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ABSTRACT 

This progress report describes the development and 
field testing of the School Mastery of Reading Test (SMRT). The SMRT 
was administered in May 1986 to 889 fourth graders in nine schools in 
Brooklyn, New York. Counts of students tested in each school and 
school profile data are provided. Machine scoring procedures are 
detailed, and results tabulated, showing that the SMRT was relatively 
easy (mean about 80% correct) with a negatively skewed score 
distribution. Subtest percentage scores are given by school and by 
district, as well as norms in percentiles and stanines. A 
professional panel of teachers, supervisors, and two Educational 
Testing Service personnel rated the usefulness of SMRT and made 
suggestions for relevant school improvement plans. To develop mastery 
scores and minimum standards, the panel judged item difficulty for 
three reading competency levels (satisfactory, minimum, below 
minimum); these judgments were coordinated with actual proportion 
correct on each item for three groups defined by Degrees of Reading 
Power test scores. The report considers the validity, reliability, 
and predictive value of SMRT, and its relation to National Assessment 
of Educational Progress norms. Attachments include questionnaires 
administered to the expert panel and the test-site teachers. (LPG) 
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EXECUTIVE SUMMARY OF ACCOMPLISHMENTS OF THE SMRT-STEPS PROJECT 



The following provides a brief listing of accomplishments of the 
SMRT- STEPS Project: 

Field tested the fourth- grade School Mastery of Reading 
Test (SMRT) in a total of nine Comprehensive Assessment 
Report (CAR) elementary schools in three Community 
School Districts. 

— Prepared a test administration manual and procedures 
which were used successfully by regular fourth grade 
teachers r with no advanced test administration training. 

— Demons tr ated that SMRT could be administered cost- 
effectively by developing re-usable test booklets and 
machine s'cannable answer sheets. 

— with the Educational Testing Service (ETS) in Princeton^ 
New Jersey^ began the development of a. high-quality 
funding proposal for potential submission to federal, 
state government and private foundations. 

— Conferred with ETS regarding future possibilities of 
using state-of -^the-art computer technology in the 
administration, scoring, reporting and storing of SMRT. 

~ E s t ab 1 ished a Professional Panel comprised of both 
teachers and supervisors to review SMRT for potential 
bias and to provide judgments related to mastery 
criteria. In addition, obtained panel member opinions 
regarding the usefulness of standardized tests and the 
types of test scores found most useful. The overall 
objective of the Professional Panel is to increase the 
meaningf ulness and instructional usefulness of all 
aspects of SMRT-STEPS. 

— Field practitioners including both teachers who 
administered SMRT and Professional Panel members 
provided ratings and opinions reflecting very favorably 
on the potential usefulness of SMRT. 

— Demonstrated that SMRT results from a mid-May 
administration are relatively high and result in the 
type of test score distribution which would be expected 
from a mastery test which is related to curriculum and 
is administered at. the end of the academic year. 



Established a prototype of SMRT New York City norms by 
generating SMRT raw score to percentile and stanine 
norms* 

Reviewed all SMRT items and found them related to New 
York City reading curriculum* 

Provided whe basis for comparing SMRT to the National 
Assessment of Education Progress ( NAEP ) by embedding 
NAEP items within SMRT and administering both at the 
same time to the same students* This may result in the 
capability of interpreting SMRT results using NAEP 
national norms and, possibly, of using NAEP as a cost- 
effective source of new test items for SMRT. 

Established the basis for comparing SMRT results to both 
Degrees of Reading Power (DRP) test and Metropolitan 
Achievement Test . (MAT) performance of grade four 
students* 

Conferred with School Im^provement Program (SIP) staff 
regarding the relationship between SMRT-STEPS and 
current New York City school improvement efforts. 
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!• BRIEF PROGRAM DESCRIPTION 



The primary objective of this project Is to develop a system to provide school 
administrators and tHchers with reading performance scores and Information useful for 
Improving the school Instructional program* Furthermore, It Is our Intention to review 
various current New York City School Improvement plans and to assess the potential 
linkage betwe^ii school level diagnosis and prescription In order to enhance the progress 
of school s. This system Is expected to be a particularly useful adjunct to the New York 
State Comprehensive Assessment Report (CAR) by diagnosing school needs for particular 
Improvement plans developed by the New York City Board of Education* 

Consequently, when SMRT-STEPS (pronounced "SMART STEPS") Is completely validated It 
may be considered a school level diagnostlc-prescrlptlve system* In effect, weaknesses 
requiring remediation will be Identified* Subsequently, results from testing may 
"elicit" or assist In the selection of school Improvement plans designed to Improve the 
effectiveness of the Instructional program In those grades for which It Is available* 

To expedite communication, the acronym "SMRT*STEPS" will be used to refer to the 
ent1i*e School Mastery of Reading Test System to Enhance Progress of Schools* The 
acronym "SMRT" will be used to refer primarily to the assessment component School 
Mastery of Reading Test* 

To enhance Its relevance and usefulness for Improving Instruction, SMRT Is being 
developed as an objective test cf mastery of reading, rather than as a norm-referenced 
test* As such, SMRT Is being designed to Indicate the extent to which specific reading 
skills have been mastered, rather than to differentiate or discriminate between 
children* Consequently, resulting subtest scores will reflect mastery or competence* 
This Is In contrast to norm-referenced scores such as grade equivalents, normal curve 
equivalents (NCE^s) and percentiles which can be misleading and are susceptible to 
misinterpretation* It Is proposed that the SMRT mastery scores Identify separately 
reported and potentially diagnostiv dimensions Including word attack, word meaning, 
literal comprehension, and reasoning comprehension* 

The currently available partially validated research version of this test Is 
designed to Identify reading subtest areas In which either small Instructional groups. 
Intact classes or the entire grade four In particular schools ^r3 not achieving 
mastery* The short-term objective Is to develop an Instructlonally useful grade four 
reading test* After Its usefulness In grade four has been established, the development 
of either a grade three or grade seven school mastery of reading test will be considered* 
It Is anticipated that fall -administered SMRT tests would be most useful to schools for 
Instructional purposes* It would be possible, also, to administer SMRT at various 
subsequent times throughout the school year to assess progress* 

Need for SMRT-STEPS is particularly timely In light of requirements of Part 100 of 
Commissioner of Education Regulations (New York State Education Department, 1984, 
1985)* These regulations Initiate an Innovative Comprehensive Assessment Report (CAR) 
which summarizes state testing program results. In addition to other school data (e*g* 
enrollment numbers, graduation results, attendance and dropout rates)* Based upon the 
''UR, 393 New York City schools (237 elementary, 102 Junior high/Intermediate and 54 high 
schools have been Identified by the New York State Education Department as In need of 
Improvement* 



II. UNIQUE AND INNOVATIVE ASPECTS 
OF THE SCHOOL MASTERY OE^ READING TEST 
SYSTEM TO ENHANCE THE PROGRESS OF SCHOOLS 



This school improvement system will be characterized by the 
following unique and innovative aspects: 

1) It is being developed by a consortium comprised of the New 
York City Board of Education and the Educational Testing 
Service of Princeton^ New Jersey* In addition to providing 
a technically sound and useful system, the p;iblic schools 
will not have to pay royalties to a test p;iblisher for the 
diagnostic part of the system* 

2) A professional panel of New York City school administrators, 
teachers, reading experts and curriculum specialists has 
reviewed SMRT for appropriateness, usefulness and potential 
bias* This panel will continue to be involved in the 
program in order to review and establish the relationship 
between assessment and school improvement materials, plans 
and programs* 

3) Common scaling between SMRT and National Assessment of 
Educational Progress (NAEP) is being established. It is 
anticipated that this will provide both cost-effective 
access to NAEP items and interpretation of SMRT results with 
respect to NAEP national norms* 

4) The diagnostic component provides an objective test of 
mastery rather than a nojrm-ref er^^nced test. As such, it is 
designed to assess reading proficiency and provides a 
relatively sensitive measure of instruction* , 

5) The diagnostic component p'rovides instructionally useful 
subscale scores to identify specific reading skills for 
diagnostic-prescriptive school improvement purposes. 

6) It is our intention to assess the feasibility of employing 
advanced computer technology and state-of-the-art psycho- 
metric techniques in the development and delivery of the 
diagnostic component and, also, in the linkage between 
assessment and school improvement materials, plans and 
programs * 

7) It is our eventual intention to design meaningful and useful 
reports of test results* Furthermore, the feasibility of 
relating subtest score profiles to prescriptive choices or 
menus of school improvement materials, plans and programs 
will be explored. 
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III . THE SCHOOL MASTERY OF READING TEST (SMRT) 



The priimry short term objective is to develop SMRT as a standardized measure of 
reading performance which can be readily adninistered and scored on a large scale and 
which accurately reflects multiple skills involved in reading. During May, 1986 SMRT 
•was adninistered to 889 students. 

SMRT is divided into two 50 item parts adninistered with a brief intervening 
intermission. Approximately 34 minutes are required for administration of each of the 
two parts for a total testing time of approximately 68 minutes. In general, there was 
sufficient time for most students to finish. In the future, however, serious 
consideration may be given to adninistering SMRT as a "power" rather than as a "speed" 
test. 

The current 100 item research version of SMRT consists of four subtests including: 
Word Attack (18 items). Word Meaning (21 items). Literal Comprehension (31 items) and 
Reasoning Comprehension (27 items). When scored, SMRT provides four subtest scores and 
one total test score. Three additional Word Recognition items appear at the beginning 
of the test. These low difficulty items are used to orient students to test directions, 
format and the separate answer sheet. In addition, th^ begin students on a positive 
note in that they are relatively easy items. It is noted that there are a small number 
of additional SMRT items, including some cloze comprehension items, which were 
eliminated from the current test in order to limit the amount of time required for test 
administration. These additional items remain part of the available item bank. 
Descriptions of the different categories of items are provided in Attachment # 1. 

In the current 100 item test, three items are used as "examples" to illustrate 
directions. These include two Word Recognition and one Word Attack item. In effect, 
students are told the correct answer after they attempt to respond. 

Directions for the most part were read to students. Incorporated within the 
remaining 97 items are 15 items obtained from the National Assessment of Education 
Progress (NAEP). Of these 16, ten are Literal Comprehension and the remaining six are 
Reasoning Comprehension items. The reasons for imbedding NAEP items within SMRT are 
discussed later in the section entitled: "National Assessment of Education Progress and 
The School Mastery of Reading Test." 

The SMRT booklet is not comprised of clearly defined subtests. Rather, items from 
the various subtests appear in both parts one and two. Furthermore, the actual tasks 
required of students change frequently. 
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IV.- TEST ADMHISTRATIOM 



Three Schools in each of three Brooklyn Community School Districts 
participated in the testing pronram. Each school previously had been identified by 
the New York State Education Department's Comprehensive Ass'.ssinent Report as in 
need of iirprovetnent based upon their grade three Degrees of Reading Power test 
results. All participating schools were selected from within the borough of 
Brooklyn for logistical, control and test security reasons. 

Profiles of each of the nine participating schools are presented in 
Table 2. Included in these profiles are sumtraries of student and 

school characteristics for the following three school years: 1282-1583; 1983-1984; 
1984-1985 

paring the second and third weeks of Kay 1986, a total of 889 fourth grade 
students were tested. As depicted in Table 1, these students represented 
approximately 90% of those for vihm answer sheets were provided. In Table 1, 
"number of Absentees" refers to those students for whom identi.ication information 
v/as provided, but item responses did not appear on their respective answer sheets. 
In other words, those students appear to have been absent for the test 
administration. 

Schools were requested to provide answer sheets for every student who was 
eligible for the annual citywide reading test, with the exception of those Special 
Education students for whom some testing variance (e.g. large print, extended time 
limits, etc.) was required. Limited English Proficient students exempted from the 
annual citywide reading test also were exempted from SMRT. In order to minimize 
disruption of instruction, provision was not made for "irake-up" testing of 
absentees. 



Test booklets and administration manuals were delivered and retrieved from all 
nine participating schools by Functional Harketing Service, the same company that 
transports citywide test materials. The following schedule was followed: 

1) Konday, May 12 - Cartons were obtained from 110 Livingston 

Street, Room 714 and delivered the same day 
to all nine schools. 

2) Friday, flay 16 - Cartons were retrieved from three schools in 

district #17 and another three schools in 
district #19 and subsequently delivered to 
110 Livingston Street, Room 714. 

3) Friday, May 23 - Cartons were retrieved from three schools 

in district #21 and delivered to 
110 Livingston Street, Room 714. 

To ensure test security, each test administration manual and test booklet was 
stamped with a unique identification number (See Attachment # 2). Careful track 
was kept of the range of numbers on both administration manuals and test booklets 
delivered to, and retrieved from, every school. 
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All test paterials v/ere delivered in strong cartons carefully sealed with 
white tape with the followino messaqe In red letters: "SECURE TEST NATEPIALS - HO 
MOT OPEN." Cartons were delivered directly to the Principal's office and receipts 
were signed. The sealed cartons were then placed in secure storage closets, 
usually'in the Principal's office. 

On the day of testing, program staff visited each school. These visitors 
retrieved the sealed cartons from locked closets and distributed test materials to 
participating classes. A careful accounting was maintained of the quantity and 
identification numbers of both test administrative manuals and test booklets 
delivered to, and subsequently retrieved from, each class. 

Program staff monitored the test administration in each school. All tests 
were adninistered by fourth grade teachers using the test administration manuals 
prepared for that purpose. Appropriate signs (See Attachment # 3 for sample) were 
placed on the door of each class indicating that "Testing" was being conducted. 
Students read the test questions from their test booklet, then responded on the 
separate answer sheet (See Attachment # 4) provided for that purpose. After 
testing, each teacher was asked to complete a one-page survey (See Attachment # 5) 
designed to assess their opinions of the test and testing procedures. 
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Table 1 



NUMBER OF FOURTH GRADE STUDENTS TESTED 





ccinnjNiTY 

SCHOOL 
DISTRICT 


SCHOOL 
NUMBER 


DATE 
TESTED 


NUMBER 
OF ANSWER 
SHEETS 


NUf-IBER 
OF 

ABSENTEES 


PERCENTAGE 
OF STUDENTS 
TESTFP 


PEPCENTAGE 
OF 

ABSENTEES 


17 


191 
289 
398 


5/13/86 
5/14/86 
5/13/86 


116 
131 
• 176 


6 

IZ 
11 


95% 

94X 


6% 


District 17 Subtotals: 




423 


29 


94X 


6% 


19 


213 

290 
223 


5/14/86 
5/15/86 
5/15/86 


111 
114 
70 


14 

n 

8 


89% 

m 

9CX 


m 

9?. 
10% 


District 19 


Subtotals: 




295 


33 


90% 


10% 


21 


90 
212 
329 


5/21/86 
5/22/86 
5/21/86 


53 
70 
48 


14 
12 
9 


79% 
85% 
84% 


21% 

15%" 

16% 


District 21 


Subtotals: 




171 


35 


83% 


17% 


Totals: 






889 


97 


90% 


10% 



0294U/6 



SCHOOL PROFILES 



Definitions for Table 2 

Listed below are definitions of the sub-headings used in 
the tables on the following pages (source:' Schoor Profiles, 
New York City Board of Education) 



Admissions - The number of pupils admitted into a school during the 
1984-1985 school year divided by the average daily register for 
this year. 

D ischarges (Departures) - The number of pupils leaving a school 
during the 1984-1985 school year divided by the average daily 
register for that year. 

Limited English Proficiency - This number is based on the Language 
Assessment Battery; if this is not available it is based on 
school entitlement. It is the number of Limited English 
Proficient students divided by the register, multiplied by 100. 

Special Education - The number of Special Education students divided 
by the register, multiplied by 100. 

Utilization - A measure of t:he usage of a school building in relation 
to it's rated capacity. 

Percent Attendance (Percent of Average Daily Register) - The mean 
pupil attendance rate for the 1984-1985 school year. 

Degrees of Reading Power Test (DRP) - The official citywide reading 
test for grades Three through Nine beginning April 1986. 

California Achievement Test (CAT) - The official citywide reading 
test from 1978 to 1985. 



■"see, for example. New York City Board of Education School 
Profiles; 1982-1983. Brooklyn, New York, 1984. 
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Table 2 



SCHCX)L PROFILES 
1982 - 1983 SCHOOL YEAR - PERCENT 



community 
School 

District School 



Limited 
English 
Profi- 

Admissions Discharges ciency 



Special 
Education 



Utilization Attendance 



17 


P.S. 


191 


59.0 


38.7 


0.0 


9.9 


100 


81.9 


17 


P.S. 


289 


44.3 


28.1 


0.1 


12.7 


87.0 


86.0 


17 


P.S. 


398 


40.5 


21.8 


2.7 


4.6 


95.0 


88.4 


19 


P.S. 


213 


48.7 


33.1 


11.5 


5.9 


97.0 


88.4 


19 


P.S. 


290 


43.7 


28.1 


21.2 


0.0 


114.0 


87.0 


19 


P.S. 


328 


37.6 


24.7 


22.1 


17.9 


60.0 


87.2 


21 


P.S. 


90 


42.7 


26.8 


3.5 


11.7 


63.0 


86.9 


21 


P.S. 


212 


24.8 


24.8 


5.3 


7.7 


78.0 


88.3 


21 


P.S. 


329 


44.7 


29.8 


14.4 


13.5 


57.0 


86.6 



19 



20 



Table 2 continued. 



SCHOOL PROFILES 



1982 - 1983 SCHOOL YEAR - PERCENT AT & ABOVE GRADE LEVEL 



DEGREES OF READING 
POWER TEST 



Conununity 
School 

District School 



( Grades ) 
Three Four 



Six 



CALIFORNIA ACHIEVEMENT TEST 



Two 



( Grades ) 
Three Four 



Five Six 



All 
Grades 



17 


P.S. 


191 


22.9 


23.4 


22.8 ! 


17.6 


20.9 


25.0 


27.2 


30.2 


23.0 


17 


I'.S. 


289 


48.0 


38.5 




21.2 


42.9 


28.6 


60.3 




37.4 


17 


P.S. 


398 


30.1 


40.6 


43.6 : 


48.6 


42.1 


39.3 


36.1 


44.7 


42.4 


19 


P.S. 


213 


53.8 


53.5 


53.8 : 


: 32.1 


54.0 


40.8 


56.6 


55.9 


47.4 


19 


P.S. 


290 


43.8 


52.0 




28.1 


50.0 


41.3 


54.1 




42.3 


19 


P.S. 


328 


26.3 


42.6 


30.3 : 


i 29.6 


23.4 


20.3 


24.2 


30.6 


25.8 


21 


P.S. 


90 


53.6 


61.9 


61.7 : 


I 32.3 


43.4 


56.9 


51.7 


61.5 


48.8 


21 


P.S. 


212 


39.6 


52.4 


74.2 


40.5 


44.1 


49.1 


65.4 


70.7 


53.0 


21 


P.S. 


329 


53.4 


60.0 


25.4 : 


: 31.6 


62.5 


45.8 


56.9 


37.7 


47.9 



I 
I 



01 



22 



erJc 



Table 2 continued. 



SCHOOL PROFILES 
1983 - 1984 SCHOOL YEAR - PERCENT 



Community 
School 

District School 



Limited 
English 
Profi- 

Admissions Discharges ciency 



Special 
Education 



Utilization Attendance 



17 


P.S. 


191 


46.1 


34.0 


2.1 


11.6 


92.0 


81.9 


17 


P.S. 


289 


56.4 


28.8 


4.1 


12.8 


125.0 


86.0 


17 


P.S. 


398 


35.9 


26.2 


3.0 


4.7 


109.0 


88.4 


19 


P.S. 


213 


41.1 


26.1 


14.0 


6.8 


98.0 


88.4 


19 


P.S. 


290 


48.5 


27.1 


31.9 


0.0 


120.0 


86.1 


19 


P.S. 


328 


37.5 


24.5 


27.5 


19.2 


58.0 


86.9 


21 


P.S. 


90 


42.5 


25.7 


9.3 


9.3 


70.0 


86.9 


21 


P.S. 


212 


35.9 


20.0 


6.1 


7.9 


78.0 


88.2 


21 


P.S. 


329 


48.9 


29.5 


14.9 


11.1 


77.0 


86.6 
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Table 2 continued... 



SCHOOL PROFILES 



1983 - 1984 SCHOOL YEAR - PERCENT AT & ABOVE GRADE LEVEL 



Community 
School 

District School 



17 
17 
17 

19 
19 
19 

21 
21 
21 



P.S. 191 

P.S. 289 

P.S. 398 

P.S. 213 

P.S. 290 

P.S. 328 

P.S. 90 

P.S. 212 

P.S. 329 



DEGREES OF READING 
POWER. TEST 



( Grades ) 
Three Four 



23.3 
/.8.4 
39.8 

48.8 
46.8 
43.2 

46.3 
46.7 
44.4 



42.1 
54.3 
43.4 

44.1 
60.7 
55.7 

58.5 
54.9 
69.6 



Six 

24.4 

43.6 

52.0 

48.2 

74.2 
78.7 
51.7 



CALIFORNIA ACHIEVEMENT TEST 



Two 

9.4 
52.8 
30.5 

31.6 
36.1 
48.3 

37.3 
53.2 
42.5 



(Grades) 
Three Four 



26.5 
35.1 
25.9 

34.1 
43.3 
26.1 

42.6 
53.3 
39.7 



35.7 
42.8 
42.7 

45.3 
53.8 
51.4 

55.9 
46.6 
39.2 



Five Six 



32.5 
73.6 
36.4 

51.2 
45.9 
29.8 

63.0 
56.9 
41.4 



40.9 

40.6 

36.2 

59.4 
68.1 
51.7 



All 
Grades 

23.4 
50.0 
34.9 

40.2 
44.6 
38.7 

51.1 
54.9 
42.7 



{ 



ERIC 



Table 2 continued.'.. 



SCHOOL PROFILES 
1984 - 1985 SCHOOL YEAR - PERCENT 



Community 
School 

District School 


✓ 

Admissions 


Discharges 


Limited 
English 
Profi- 
ciency 


Special 
Education 


Utilization 


Attendance 


17 
17 
17 


P.S. 191 
P.S. 289 
P.S. 398 


44.2 
49.2 
33.7 


25.0 
24.0 
25.0 


5.0 
4.8 
4.5 


11.8 
12.1 
4.2 


97.0 
131.0 
106.0 


85.0 
88.2 
89.5 


19 
19 
19 


P.S. 213 
P.S. 290 
P.S. 328 


42.0 
44.8 
29.9 


25.6 
32.4 
21.0 


18.2 
38.7 
28.1 


5.8 
1.3 
19.6 


97.0 
131.0 
54.0 


87.7 
87.0 
87.8 


21 
21 
21 


P.S. 90 
P.S. 212 
P.S. 329 


34.4 
32.8 
48.4 


22.6 
18.2 
26.5 


19.1 
10.1 
24.3 


9.1 
5.9 
11.5 


65.0 
74.0 
78.0 


87.4 
89.4 
87.1 



• I 



Table 2 continued... 



SCHOOL PROFILES 



1984 - 1985 SCHOOL YEAR - PERCENT AT & ABOVE GRADE LEVEL 



DEGREES OF READING 
POWER TEST 



Conmiinity 



District 


School 


Three 


Four 


Six : 


Two 


Three 


Four 


Five 


Six 


. 17 


P.S. 


191 


49.3 


43.6 


49.1 : 


3 :.8 


36.7 


43.9 


35.0 


56.0 


: 17 


P.S. 


289 


47.7 


53.2 




59.1 


58.7 


50.8 


80.2 




17 


P.S. 


398 


42.5 


55.9 


46.7 ! 


! 33.3 


45.2 


44.1 


47.1 


52.0 


19 


P.S. 


213 


47.1 


59.6 


30.0 ! 


: 30.9 


38.6 


53.6 


58.7 


56.1 


■ 19 


P.S. 


290 


41.4 


58.3 




23.0 


44.4 


48.7 


60.2 




19 


P.S. 


328 


48.1 


49.3 


37.7 ! 


; 33.8 


47.0 


34.8 


55.8 


22.2 


21 


P.S. 


90 


56.7 


50.9 


60.7 


: 37.3 


33.9 


56.5 


60.0 


67.3 


21 


P.S. 


212 


63.0 


43.5 


72.0 


45.9 


40.5 


46.6 


60.0 


71.8 


. 21 


r.s. 


329 


57.1 


42.1 


42.9 


: 25.8 


38.6 


38.1 


57.4 


42.6 



CALIFORNIA ACHIEVEMENT TEST 



(Grades) 



39.7 
61.8 
43.5 

46.7 
42.3 
38.9 

50.3 
53.8 
39.7 
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V. MACHINE SCORING OF STUDENT ANSWER SHEETS 

Student answer sheets were scanned and scored by the 
Office of Educational Assessment Scan Center at 49 Flatbush 
Avenue, Brooklyn, New York. The machine scannable general 
purpose NCS answer sheets (i.e., NCS Trans-Optic EB08- 
4521:223222) were scanned on an NCS 7018 Optical Mark Reader 
with NCS Scanpak "Test Scoring Package" software. 

To ensure the accuracy of results, the following manual 
quality control procedures were instituted : 

Preparation of Machine Readable Answer Sheets 

1. Inserted a school code for each answer sheet 

2. Separated the answer sheets of the absentees 

3. Checked the Name, NYC Identification Number, Date of 
Birth, and Class Number for completeness. In the 
instances where the nine digit NYC student Identification 
Numbers on the answer sheet was either incomplete, 
missing or was not the same as that on the Office of 
Student Information Services (OSIS) Student Bio-File 
Roster, the school was telephoned. In every instance, 
the identification number accepted as final was that 
provided by the school. However, for a small number of 
students who were either in Promotional Gates classes or 
whose name did not appear on the OSIS Student Bio-File 
roster, it was not possible to cross-check the accuracy 
of the identification number corresponding to the names. 

4. For all the items mentioned in the previous step, 
verified that the circles corresponded with the 
characters 

5. Tidied answer sheets - 

a. Erased unnecessary marks 

b. Shaped circles that extended to other areas 

c. Darkened light circles 

d. Identified and rectified circles which were only 
partially filled 

e. When a light circle (e.g., erased circle) was 
adjacent to a dark circle (intended circle) the 
circles were lightened and darkened respectively 

f . Any torn answer sheet was replaced by a fresh answer 
sheet, appropriately completed 

Scanning 

1. To evaluate the performance of the scanner under varied 
conditions, two dummy answer sheets were prepared and 
scanned. The features of the dummy sheets were as follows: 
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SHEET 1 



SHEET 2 



Clean sheet 



Untidy sheet 



Included answer choice "e" Included answer choice "e" 



2. The results of the dummy answer sheets and one additional 
answer sheet were hand-scored and compared with the 
score obtained by scanning. 



Scan Tape and Printout 

A printout was produced from the magnetic tape 
resulting from the scanning procedure. This printout was 
reviewed to be certain that : 

a. Both schools and classes were grouped together^ as 
required. 

b. The name^ identification number appeared to be complete 

c. The date of births identification number and class number 
appeared in the required field 



Computer Scoring 

1. The answers that were scanned and stored on magnetic tape 
as a data file^ were read and scored using a SPSS-X 
program which generated results in the form of Five 
Subtest Scores and a Total Test Score. These results were 
compared with the scores generated by the scanning 
program. 

2. In running the SPSS-X program, a few errors were detected 
in the form of incomplete names, identification numbers, 
dates of birth and class numbers. This provided a final 
check and improved the completeness of the biographical 
data fields. 



Included missing values 
Included multiple answers 



Included missing values 
Included multiple answers 
Simulated erased circles 
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VI. RESULTS FROM THE SPRIHG 1986 SCHOni MASTERY OF PEmm TEST 



Preliminary SMRT descriptive statistics from the fay 1986 test 
administration are presented in Table 3. Total test score suirmaries are 
presented for the nine participating schools In three school districts. These 
results are depicted also In Figures 1 and 2. Specifically, Figure 1 depicts 
percentages for all nine schools. Figure 2 depicts percentages for the three 
school districts. Results for each of the subtests. In addition to the total 
test, are presented In Table 4 . In addition to the sumrary statistics, it Is 
noted that additional Individual student listings Including subtest and total 
test scores were generated by class and grade. 

The overall mean total test score was 80.54 with a standard deviation of 
13.49. The overall median total test score was 84 with an intor-quartlle 
range of 16. In effect, these statistics reflect a negatively skewed 
distribution. In other words, this is a relatively easy test with a "piling 
up of scores" at the high end of the score distribution. 

This distribution v/as expected for a curriculum-based test, such as SflPT, 
administered at the end of the school year; It is likely that the relatively 
high overall scores reflect mastery, at least to some extent, of fourth orade 
curriculum taught during the school year. 

Results for both boys and girls were examined. Of the 889 students 
tested, 447 girls achieved a r:ean score cf 81.08 (with a standard deviation of 
12.78) and 442 boys achieved a mean score of 80.00 (with a s^^ndard deviation 
of 14.16) 



0294U/11 



Table 3 



SMRT SUMMARY STATISTICS 



NUMBER INTER 
DISTRICT OF MEAN STANDARD MEDIAN QUARTILE 

SCHOOL STUDENTS DEVIATION RANGE 



CSD 17 


423 


80.63 


13.53 


83 


15 


PS 191 


116 


80.96 


12.98 


84 


13 


PS 289 


131 


82.17 


13.75 


85 


14 


PS ^98 




/ 7 . A 1 


1^ • 0 / 


Q 1 


15 


CSD 19 


295 


79.05 


14.37 


83 


18 


PS 213 


111 


77.41 


15.70 


81 


22 


PS 290 


114 


78.63 


13.70 


81 


17 


PS 328 


70 


82.33 


12.80 


86 


14 


CSD 21 


171 


82.91 


11.38 


86 


13 


PS 90 


53 


85.11 


P/.81 


87 


11 


PS 212 ' 


70 


79.87 


13.57 


81 


19 


PS 329 


48 


84.90 


8.26 


87 


11 


TOTAL 


889 


80.54 


• 13.49 


84 


16 



100 



Figure 1 

PERCENTAGES OF SIfRT SCORES BY SCHOOLS 



00 - 



60 -^-0 n 
70-/s^^X 

50 -/S^^X 

/s^sx 

30 -/V^^>< 

/s^^x 

10 --^V^SX 

/sAx 
0^ 



191 



289 



398 



Key: 



WORD 
ATTACK 



WORD 
MEANING 



213 



< 
< 
< 
< 
< 
< 
< 
< 
< 
< 
< 
< 
< 
< 

X 

X 



PI 



/SWSX 

4sKlsx 

/v^x 
/vWsx 
/vgisx 
/sWsx 
/v^x 
/sHsx 



290 



A 



X 



/vgRx 
/sHsx 

/s^x 
/s^x 
/s^x 

/v^X 

'f^wJx 

aHsx 

/SMSX 

/N^X 

/s^x 
/^x 



328 



Pra n 

/s^ X 
/s^^x 

i. 



90 



/'VWSX 



/v^x 

^ wX 
/sKSx 

/vHsx 
/sWsx 
/swsx 
/s^x 



212 



/ 



329 



PiihHn Schftfil Nutwiiwir 



LITERAL REASONING TOTAL 

COMPREHENSION COMPREHENSION 



I 



100 



Figure 2 

PERCENTAGES OF SMRT SCORES BY DISTRICTS 
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Table 4 



SPRING 1986 SHRT RAW SCORE RESULTS 
(Number of Students = 889) 



WORD 
RECOGNITION 
(1=3) 



WORD 
AHACK 
(1=18) 



WORD 
hSANING 
(1=21) 



LITERAL 
COMPREHENSION 
(1=31) 



REASONING 
COMPREHENSION 
(1=27) 



TOTAL 
SCORE 
(1=100) 



MEAN 

STANDAHO 
DEVIATION 



2.97 
0.20 



15.09 
2.90 



17.02 
3.55 



25.49 
4.58 



19.97 
4.25 



80.54 
13.49 



I 

ro 
o 
I 



MEDIAN 3 

INTER- 
QUARTILE RANGE 0 



16 
3 



18 
5 



27 
6 



21 
5 



84 
16 



RANGE 

MAXIMUM 

MINIMUM 



3 
3 
0 



16 
18 

2 



21 
21 

0 



31 
31 
0 



27 
27 

0 



94 
99 
5 
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VII. NORMS TABLE FOR THE SPRING 1986 SCHOOL MASTERY OF READING TEST 



Table 5 provides an example of "local" SMRT norms which 
eventually could be developed for New York City. This table was 
based upon student scores from all participating nine schools and 
was developed for illustrative purposes. It is important to note 
that percentile scores are most appropriate for norm-referenced 
interpretations of test results. The primary intention is to 
develop SMRT as a test which reflects mastery or competence 
rather than as a norm-referenced test. Primary effort, 
therefore f is being placed on developing mastery scores. 
Percentiles may be used, subsecjuently, in a secondary or 
ancillary capacity to mastery scores. 

Table 5 provides the basis for any subtest or total test raw 
score (i.e., number of correct test items) to be interpreted 
relative to overall group performance. In other words, any SMRT 
subtest or total test raw score can be converted into either a 
percentile or stanine by using this "raw score to percentile rank 
and stanine norms" taQ^le. 

Percentiles allow us to compare any student's perfoxnnance at 
a given time with that of the overall group on a scale ranging 
from 1 to 99. On this scale, the fiftieth percentile, or median, 
might be considered "average." Use of this table may be 
illustrated by locating a raw score of 90. This raw score 
corresponds to the seventy-sixth percentile. This seventy-sixth 
percentile means that the student's score exceeded those scores 
of approximately 76 percent of the noxnn group for that time of 
the year. In this case, the norm group is comprised of 889 
students who were administered SMRT during May 1986 in nine New 
York City public schools located in three Community School 
Districts. 

For total test scores, percentiles may be instruct ionally 
useful and assist in the interpretation of raw scores. Foi 
reporting subtest results, however, percentiles are less useful. 
In this report, percentiles have been provided for each of the 
subtests for illustrative purposes only. The use of percentiles 
for reporting subtest results is not recommended because they may 
be misleading and subject to misinterpretation. Due to the 
relatively small number of test items within any given subtest, 
the change of percentiles from one. raw score to another may be 
relatively large. Consequently, these percentiles for subtests 
may erroneously appear to reflect significant and meaningful 
differences in test performance from one raw score to an adjacent 
raw score. In reality, the difference in actual performance from 
one raw score to another may be relatively modest. For example, 
examination of the norms table for the "Word Attack" subtest 
reveals that a change in raw score from 17 to 18 correct 
corresponds to a change from the 77th to the 99th 
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percentile. In effect, a change of one raw score corresponds to 
a 22 point change in percentiles. 

This table presents, also,, stanine scores. Stanine scores 
(i.e., a contraction of "standard nine") are normally distributed 
standard scores v/hich range from one to nine. They have the 
convenience of being a single digit and are relatively easy to 
interpret. The stanine scale has a mean of five and a standard 
deviation of 1.96. 
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Table 5 



SMRT RAW SCORES, PERCENTILES AND STANINES 



STA- 
NINES 


1 

jpER- 

jCENT- 

IILES 


RAW SCORES 

1 _ 1 


i ■ 

w.R. i 


W.A. 


1 t 1 
1 1 I 

1 W.H. 1 L.C. 1 


R.C. 


I TOTAL 




! 99 


3 1 


18 


j 21 ! 31 i 


27 


1 98 


9 


i 98 i { 






26 






1 97 1 ! 




1 1 1 




1 97 




1 96 1 j 




i ! 1 
1 1 1 








I 95 1 




1 1 1 


25 






1 94 1 




1 1 1 




j 96 


8 


! 93 1 




i i 30 








I 92 1 








i 95 




1 91 1 




i i 1 








1 90 1 




! i ! 




1 94 




1 89 i 




• . • ' 








1 88 i j 




i 20 1 








1 87 1 ' 








i 93 




! 86 i 




1 i 


24 






i 85 1 




i 1 






7 


! 84 i 




i ' ! 




i 92 




1 83 1 




i 1 








1 82 1 




i i 29 








! 81 i 








! 91 




I 80 1 




1 1 








I 79 1 




i ! 


23 






1 78 1 




! 1 

! i 








1 77 1 


17 


! ! 








i 76 1 




! ! 




j 90 




1 75 1 




1 1 








1 74 1 




1 i 








1 73 1 




1 19 1 








1 72 1 








i 89 




1 71 1 




1 1 28 








1 70 1 






22 




6 


! 69 j 












1 68 i 












1 67 1 


1 






i 88 




! 66 i 












! 65 i 


1 










i 64 i 












I 63 i 








i 87 




1 62 i 












1 61 i 












1 60 1 








1 86 



W.R. = Word Recognition 
W.A.= Word Attack 
W.M.« Word Meaning 



L.C.= Literal Comprehension 
R.C Reasoning Comprehension 



Table 5 continued 



SMRT RAW SCORES, PERCENTILES AND STANINES 



{PER- 
CENT- 
ilLES 



RAW SCORES 



W.R. 



W.A. 



W.M. 



L.C. 



R.C. 



59 
58 
57 
56 
55 
54 
53 
52 
51 
50 
49 
48 
47 
46 
45 
•44 
43 
42 
41 

40 
39 
38 
37 
36 
35 
34 
33 
32 
31 
30 
29 
28 
27 
26 
25 
24 
23 

22 
21 
20 



16 



27 



21 



18 



26 



20 



15 



17 



25 



19 



14 



16 



24 



18 



15 



13 



23 



22 



17 



40 
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Table 5 continued 



SHRT RAW SCORES, PERCENTILES AND STANINES 



STA- 
NINES 


{ { 

InpO. 1 1 

1 FBR- • 1 
1 C&NT- j 
;iLES 1 W.] 




RAW SCORES 




— 1 




! 1 

1 W.M. 1 


L.C . 1 


R.C. 


1 




1 1 a 1 
1 t 




! 14 1 




1 ^ 

10 


1 71 




1 1 O 1 

1 1 


} 


1 1 






1 7n 
1 / \j 




1 1 •? 1 


1 




21 1 








1 1 


1 1 0 


i i 






i 




1 1 C 1 




1 13 1 






j oo 




1 14 ! 


1 


1 1 






1 67 




I 13 1 






20 ! 




i 66 




i 1 o t 
1 IZ 1 


1 1 1 


i j 






1 \jD 




1 111 
1 11 1 


} 








< f)^ 




i 10 i' 




1 12 1 


19 1 


14 


1 62 




! 9 ! 


j 








1 60 


2 


! 8 1 


1 10 




18 ! 


13 


1 58 




1 7 1 




i 11 i 


17 j 




1 57 




i 6 i 










! 55 




! 5 1 


i 9 


! 10 I 


16 1 


12 






! 4 1 


1 8 


1 8 1 


15 j 


11 


i 50 


1 


1 3 1 




I 7 1 




10 


1 47 




! 2 1 2 






12 j 


9 


1 43 




1 1 ! 






11 1 


8 


i 35 
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Vlll. TYPES OF SCORES USED TO REPORT TEST RESULTS 



In anticipation of the eventual need to develop a meaningful and Instructlonally 
useful method of reporting test results, the Preliminary Professional Panel was asked to 
* rate their opinion of the usefulness of the five types of test scores (see Attachment # 
61 Definitions of the five different types of test scores are provided In Attachment # 
T. A summary of these ratings Is provided In Table 6 , It Is apparent that the panel of 
professional educators considered percentile rankings as the most Instructlonally useful 
type of test score. 

In general, the types of scores referred to above reflect relative rather than 
absolute performance. In effect, they provide nonrnreferenced Interpretations of test 
results. In contrast. It Is anticipated that the most Instructlonally useful and 
appropriate type of score will reflect mastery of reading. In other words, criteria, 
standards or "passing scores" should be developed for item clusters, subtests and, perhaps, for 
the total test. Subsequently, these "benchmarks" will be used to determine If results 
reflect mastety. Such Indices reflect absolute performance and are most appropriate for 
criterion-referenced or mastery tests such as SMRT. 

It should be noted that alternate methods of reporting test results to enhance 
school progress maty be developed. These methods might, for example, be based upon the 
diagnostic-prescriptive relationship between SMRT assessment and school Improvement 
plans, programs and materials. 



0294U/10 



Table 6 



PROF£SSI(»IAL PANEL MEMBER OPINIONS OF THS USBFULNBSS 
OP VARIOUS TYPES OF TEST SCORES 





NUMBER 

OF 
RESPON- 
DENTS 


MEAN 


S.D 












PERCENTAGES OF RATINGS 
3.5 4 4.5 












LOW 
1 


1.5 


2 


2.5 


3 


5 


5.5 


6 


6.5 


HIGH 
7 


PERCENTILE RANKINGS 17 


5*65 


0.98 


0 


0 


0 


5«9 


0 


0 


0 


0 


23.5 


5.9 


52. 9 


0 


11.8 


ORADB EQUIVALENTS 


17 


5.35 


1.46 


5.? 


0 


0 


0 


0 


0 


17.6 


0 


17.6 


5.9 


29 .-4 


5.9 


17.6 


RAW SCORES 


17 


4*65 


1.45 


0 


5.9 


0 


0 


23.5 


0 


5.9 


5.9 


17.6 


11.8 


23.5 


0 


5.9 


STANINES 


17 


4.12 


1.59 


0 


0 


23.5 - 


0 


11.8 


5.9 


5.9 


11.8 


23.5 


5.9 


0 


0 


11.8 


NORMAL CURVE- 


16 


2.41 


1.28 


25 


6.3 


25 


12.5 


12.5 


0 


6.3 


0 


12.5 


0 


0 


0 


0 



EQUIVALENTS 

S8SSSSS=S=SSSS=SS=S3BSSSSSZ==SX=aSX«XSSS= 



I 

to 
i 



43 



44 
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IX. DEVELOPING MASTERY SCORES AND ESTABLISHING MINIMUM STANDARDS 

In order to establish mastery criteria for each test item, both eippirical data and 
judgments of experts are being reviewed. The eventual primary objective is to identify 
SMRT performance standards. It is likely that such standards will be based either upon 
item clusters or subtests, rather than upon individual items or total test scores. 

Empirical d&va which have been obtained include: 

1) SMRT scores of fourth graders in the nine schools tested during the second 
and third weeks in May 1986, 

Z) Degrees of Reading Power (DRP) test scores of the same students tested on 
May 7, 1986, and 

3) Metropolitan Achievement Test (MAT) reading scores of the same students 
tested on both April 21 and 22, 1986. 

Judgments of experts were provided by the Professional Panel. This group was 
comprised of professional educators including teachers, assistant principals, 
principals, district reading coordinators and curriculum supervisors. Panel members 
were asked to estimate the difficulty level of each item in the Group Test for each of 
the three hypothetical groups of students described below: 

Group 1 ; Sati sfactory readers . Students in this group 

J] read wen enough to learn from fourth grade text materiel in 
reading and other subject areas, 

b) read well enough to follow instructions in workbooks, 
arithmetic problems, and other school work, and 

c) can be expected to continue to learn in the fifth grade. 

Group 2 ^ Minimally competent readers . Students in this group 

TI have developed sufficient reading skills that they can 
continue to learn to read, perhaps with special help, 

b) can be expected to have some difficulty with fourth grade text 
material, but can learn at a minimal level from such material, 
and 

c) can be expected to need continuing special help with basic 
reading skills in the fifth grade. 

Group 3 ; Readers below minimum competence. Students in this group 

TI have not achieved some or a]i of the basic reading skills 
appropriate to fourth grade, 

b) cannot learn by reading fourth grade text material in reading 
and other subject areas, 

c) cannot read sufficiently well to follow directions in 
workbooks and arithmetic problems. 

Both empirical and judgmental information will be reviewea carefully and 
alternative strategies for setting performance standards {see, for discussion, 
Livingstrn & Zieky, 1982) will be considered. 
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Table 7 presents Professional/Panel judgments or 
expectations of performance on all SMRT items for the three 
hypothetical groups of students described earlier. In addition, 
actual performance on all SMRT items for relatively low, moderate 
and relatively high achieving groups of students, as determined 
by their DRP test scores, is presented in Table 8. The three DRP 
score categories were established based upon the grade four DRP 
Promotional Gates criterion. Students achieving DRP scores 
within one standard error either below or above the fourth grade 
criterion were assigned to Group 2. Stuc?^nts achieving DRP 
scores lower than one standard error below the Promotional Gates 
criterion were assigned to Group 3. Students achieving DRP 
scores higher than one standard error above the Promotional Gates 
criterion were assigned to Group 1. Similar groupings can be 
established based upon Metropolitan Achievement Test scores. 

Once reasonable performance expectations of items are 
established, standards for test objectives, item clusters and/or 
subtests can be set by aggregating items. 

Additional information, such as frequency distributions of 
test scores, can be useful in setting standards. For example, 
SMRT subtest and total test scores can be grouped and depicted in 
various ways. Figure 3, for example, depicts SMRT subtest and 
total test raw scores in ten groups based upon Degrees o£ Reading 
Power (DRP) test performance. The distribution of DRP scores was 
divided into ten groups ranging from the lowest to the highest 
test scores. Subsequently, SMRT subtest and total test 
performance was depicted as histogreuns which illustrate the 
profile of SMRT scores for each DRP category. Using this 
information, performance standards could be established by 
picking the point between two specific groups of SMRT histograms. 
It is noted, in addition, that it is an interesting research 
question to determine if different school improvement or 
corrective action plans can be related to profiles of scores 
obtained by students of different achievement levels. 
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Table 7 

PROFESSIONAL PANEL JUDGEMENTS OF TEST ITEM DIFFICULTY 
FOR THREE HYPOTHETICAL GROUPS OF STUDENTS 



Wiiat proportion of students do you think will get this item correct ? 



QUESTION GROUP 



All or 
almost all 
(91% or 
more) 



About About About None or 

three- half one almost 

quarters quarter none (10% 

(61-90%) (36-60%) (11-35%) or less) 



1 


Group 1 


--- 


--- 


(SAMPLE QUESTION) 


— — — 




Group 2 


— — — 


— — — 






M M M 
















2 


Group 1 


--- 


--- 


(SAMPLE QUESTION) 


— — — 




Group 2 















varoup J 












3 


Group 1 


84.2 


15 . 8 


0 


0 


0 




Group 2 . 


21 


63.2 


15.8 


0 


0 




Group 3 






52.6 


26.3 


10 5 


4 


Group 1 


100 


0 


0 


0 


0 




Group 2 


55 


40 


5 


0 


0 




V3£^OUp O 


s 


45 


25 


15 


10 


5 


Group 1 


85 


15 


0 


0 


0 




Group 2 


20 


55 


25 


0 


0 




Group 3 


5 


5 


40 


30 


20 


6 


Group 1 


95 


5 


0 


0 


0 




Group 2 




65 


10 


0 


0 




Group 3 


5 


20 


35 


30 


10 


7 


Group 1 


75 


25 


0 


0 


0 




Group 2 


20 


45 


35 


0 


0 




Group 3 


5 


20 


10 


45 


20 


8 


Group 1 


95 


5 


0 


0 


0 




Group 2 


40 


55 


5 


0 


0 




Group 3 


10 


30 


30 


20 


10 


9 


Group 1 






(SAMPLE QUESTION) 






Group 2 














Group 3 












.0 


Group 1 


50 


50 


0 


0 


0 




Group 2 


10 


30 


45 


15 


0 




Group 3 


0 


5 


15 


40 


40 



' (continued in next page) 
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Table 7 continued . . . 



PROFESSIONAL PANEL JUDGEMENTS OF TEST ITEM DIFFICULTY 
FOR THREE HYPOTHETICAL GROUPS OF STUDENTS 



What proportion of students do you think will get this 


Item 


correct ? 






All or 


About 


About 


About 


None or 






almost 


all 


three- 


half 


one 


almost 






(91% or 


quarters 




quarter 


none 


(10% 


QUESTION 


GROUP 


more) 


(61-90%) 


(36-60%) 


(11- 


•35%} 


or 


less) 






















11 


Group 1 




70 


25 


5 




0 




0 




Group 2 




10 


35 


50 




5 




0 




GrouD 3 




0 


5 


15 




45 






12 


Group 1 




80 


20 


0 




0 




0 




Group 2 




5 


50 


45 




0 




0 




Group 3 




0 


0 


15 




60 






13 


Group 1 




85 


10 


5 




0 




0 




Group 2 




25 


50 


20 




5 




0 




Group 3 




0 


5 


35 




40 






14 


Group 1 




90 


10 


0 




0 




0 




Group 2 




50 


45 


5 




0 




0 








0 


30 


40 




25 






15 


Group 1 




95 


5 


0 




0 




0 




Group 2 




15 


70 


15 




0 




0 




Group 3 




0 


10 


20 




50 




ZU 


16 


Group 1 




90 


10 


0 




0 




0 




Group 2 




25 


45 


30 




0 




0 




Group 3 




0 


20 


25 




20 




35 


17 


Group 1 




100 


0 


0 




0 




0 




Group 2 




50 


35 


15 




0 




0 




Group 3 




0 


40 


20 




30 




10 


18 


Group 1 




80 


20 


0 




0 




0 




Group 2 




20 


60 


15 




5 




0 




Group 3 




5 


10 


35 




30 




20 


19 


Group 1 




95 


5 


0 




0 




0 




Group 2 




25 


70 


5 




0 




0 




Group 3 




0 


5 


50 




35 




10 


20 


Group 1 




55 


45 


0 




0 




0 




Group 2 




5 


30 


50 




15 




0 




Group 3 




0 


5 


15 




25 




55 
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Table 7 continued . . . 



PROFESSIONAL PANEL JUDGEMEIWS OF TEST ITEM DIFFICULTY 
FOR THREE HYPOTHETICAL GROUPS OF STUDENTS 



What proportion of students do you think will get this item correct ? 



All or About About About None or 

almost all three- half one almost 

(91% or quarters quarter none (10% 

QUESTION GROUP more) (61-90%) (36-60%) (11-35%) or less) 



21 


Group 1 


90 


10 


0 


0 


0 




Group 2 . 


25 


55 


20 


0 


0 




Group 3 


5 


10 


15 


55 


15 


22 


Group 1 


95 


5 


0 


0 


0 




Group 2 


45 


40 


15 


0 


0 




Group 3 


5 


15 


25 


40 


15 


23 


Group 1 


40 


60 


0 


0 


0 




Group 2 


5 


30 


55 


10 


0 




Group 3 


0 


0 


10 


20 


70 


24 


Group 1 


55 


45 


0 


0 


0 




Group 2 


5 


30 


55 


10 


0 




Group 3 


0 


5 


10 


35 


50 


25 


Group i 


85 


15 


0 


0 


0 




Group 2 


25 


60 


10 


5 


0 




<Jroup 3 


5 


5 


35 


45 


10 


26 


Grou;: 1 


80 


20 


0 


0 


0 




Group 2 


5 


55 


30 


10 


0 




Group 3 


0 


15 


10 


40 


35 


27 


Group 1 


70 


30 


0 ' 


0 


0 




Group 2 


15 


40 


45 


0 


0 




Group 3 


5 


10 


20 


25 


40 


28 


Group 1 


85 


15 


0 


0 


0 




Group 2 


20 


35 


45 


0 


0 




Group 3 


5 


5 


30 


40 


20 


29 


Group 1 


80 


20 


0 


0 


0 




Group 2 


20 


40 


40 


0 


0 




Group 3 


0 


10 


35 


30 


25 


30 


Group 1 


95 


5 


0 


0 


0 




Group 2 


SO 


40 


5 


5 


0 




Group 3 


5 


30 


30 


20 


15 



o 

FRIC 



(continued i. next page) 
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Table 7 continued... 



PROFESSIONAL PANEL JUDGEMENTS OF TEST ITEM DIFFICULTY 
FOR THREE HYPOTHETICAL GROUPS OF STUDENTS 



What proportion of students do you think will get this item correct ? 



QUESTION GROUP 



All or 
almost all 
(91% or 
more) 



About About About None or 

three- half one almost 

quarters quarter none (10% 

(61-90%) (36-60%) (11-35%) or less) 



31 


GroiiD 1 


95 


5 


0 


0 


0 




Group 2 


50 


40 


10 


0 


0 




Group 3 


0 


35 


30 


25 


10 


32 


flrouD 1 


60 


40 


0 


0 


0 




Group 2 


0 


35 


55 


10 


0 




Group 3 


0 


0 


15 


30 


55 


33 


flrouo 1 

w VA ^ ^ 


75 


20 


5 


0 


0 




•Group 2 


25 


40 


25 


10 


0 




Group 3 


5 


10 


20 


35 


30 


34 




65 


35 


0 


0 


0 




Group 2 


5 


45 


25 


25 


0 




Group 3 


0 


0 


10 


20 


70 


35 


GTOMTi 1 


36.8 


63.2 


0 


0 


0 




Group 2 


0 


15.8 


■ 68.4 


15.8 


0 




Group 3 


0 


0 


0 


26.3 


73.7 


36 


Group 1 


55 


45 


0 


0 


0 




Group 2 


0 


35 


50 


15 


0 




Group 3 


0 


0 


5 


35 


60 


37 


Group 1 


55 


40 


5 


0 


0 




Group 2 


5 


35 


45 


15 


0 




Group 3 


0 


10 


5 


40 


45 


38 


Group 1 


30 


60 


10 


0 


0 




Group 2 


5 


10 


55 


30 


0 




Group 3 


0 


0 


5 


25 


70 


39 


Group 1 


65 


35 


0 


0 


0 




Group 2 


20 


15 


55 


10 


0 




Group 3 


C 


5 


5 


40 


50 


40 


Group 1 


50 


45 


5 


0 


0 




Group 2 


0 


35 


55 


10 


0 




Group 3 


0 


0 


5 


40 


55 
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Table 7 continued . . . 

PROFESSIONAL PANEL JUDGEMENTS OF TEST ITEM DIFFICULTY 
FOR THREE HYPOTHETICAL GROUPS OF STUDENTS 



What proportion of students do you think ^ill get this item correct ? 







All or 


About 


About 


About 


None or 






almost all 


three- 


half 


one 


almost 






(91% or 


quarters 




quarter 


none (10% 




GROUP 


more ) 


(61-90%) 


(36-60%) 


(11-35%) 


or less 


41 


Group 1 


65 


30 


5 


0 






Group 2 


20 


35 


20 


25 


0 




Group 3 


5 


10 


10 


20 


55 


42 


Group 1 


80 


20 


0 


0 


A 




Group 2 


35 


35 


30 


0 


0 




Group 3 


5 


10 


30 


30 


25 


43 


Group 1 


85 


15 


0 


0 


A 




Group 2 


45 


30 


25 


0 


0 




Group 3 


5 


20 


20 


35 


20 


44 


Group 1 


60 


40 


0 


0 


0 




Group 2 


15 


45 


25 


15 


0 




Group 3 


0 


15 


15 


30 


40 


45 


Group 1 


70 


30 


0 


0 


0 




GrouD 2 


30 


25 


35 


10 


0 




GrouD 3 


0 


15 


15 


35 


35 


46 


Group 1 


65 


35 


0 


0 


0 




Group 2 


25 


35 


35 


5 


0 




GiTOUD 3 


0 


10 


1».0 


30 


40 


47 


Group 1 


30 


65 


5 


0 


0 




Group 2 


0 


25 


55 


20 


0 




Group 3 


0 


0 


0 


20 


80 


48 


Group 1 


30 


70 


0 


0 


0 




Group 2 


0 


30 


50 


20 


0 




Group 3 


0 


0 


0 


25 


75 


49 


Group 1 


52.6 


42.1 


5.3 


0 


0 




Group 2 


0 


31.5 


63.2 


5.3 


0 




Group 3 


0 


0 


0 


21.1 


78.9 


50 


Group 1 


35 


65 


0 


0 


0 




Group 2 


0 


20 


65 


15 


0 




Group 3 


0 


0 


0 


20 


80 
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Table 7 continued . . . 

PROFESSIONAL PANEL JUDGEMENTS OF TEST ITEM DIFFICULTY 
FOR THREE HYPOTHETICAL GROUPS OF STUDENTS 



What proportion of students do you think will get this item correct ? 









All or 


About 


ADOUt 


ADOUt 


None or 








almost all 


three- 


half 


one 


almo;vst 








(91% or 




quarters 




quarter 


none (ll)% 


fESTION 


GROUP 


more) 






\ J0-"00^ ) 


1 11-JD^ ) 


or le^is 


51 


Group 


1 




75 


25 


0 


0 


0 




Group 


2 




30 




ZD 


c 
D 


A 

u 




Group 


3 




5 


1 c 

Id 


Id 






52 


Group 


1 




80 


20 


0 


0 


0 




Group 


2 




35 


o n 
ZU 


OD 


1 n 
10 


A 
U 




Group 


3 




5 


Id 


zO 


A Q 

4d 


ID 


53 


Group 


1 




60 


35 


5 


0 


0 




Group 


2 




15 


40 


zO 


ZD 


A 

U 




Group 


3 




0 


10 


Id 


ZD 


DU 


54 


Group 


1 




45 


50 


5 


0 


0 




Group 


2 




0 


40 


A C 

4d 


Id 


A 

u 




Group 


3 




0 


0 


c 
D 


40 


DD 


55 


Group 


1 




65 


30 


5 


0 


0 




Group 


2 




20 


iD 


40 


D 


A 

u 




Group 


J 




0 


0 


10 


4d 


A C 

4d 


56 


Group 


1 




85 


15 


0 


0 


0 




Group 


2 






25 


40 


0 


A 




Group 


3 




5 


15 


15 


40 


ZD 


57 


Group 


1 




65 


35 


0 


0 


0 




Group 


2 




15 


40 


35 


10 


0 




Group 


3 




0 


g 


30 


35 


35 


58 


Group 


1 




65 


35 


0 


0 


0 




Group 


2 




0 


55 


25 


20 


0 




Group 


3 




0 


0 


20 


25 


55 


59 


Group 


1 




85 


15 


0 


0 


0 




Group 


2 




15 


60 


20 


5 


0 




Group 


3 




5 


10 


10 


45 


30 


60 


Group 


1 




85 


15 


0 


0 


0 




Group 


2 




35 


35 


30 


0 


0 




Group 


3 




0 


20 


5 


50 


25 
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Table 7 continued... 



PROFESSIONAL PANEL JUDGEMENTS OF TEST ITEM DIFFICULTY 
FOR THRSE HYPOTHETICAL GROUPS OF STUDENTS 



QUESTION GROUP 



61 



62 



63 



64 



65 



66 



67 



68 



69 



70 



Group 1 
Group 2 
Group 3 

Group 1 
Group 2 
Group 3 

Group 1 
Group 2 
Group 3 

Group 1 
Group 2 
Group 3 

Group 1 
Group 2 
Group 3 

Group 1 
Group 2 
Group 3 

Group 1 
Group 2 
Group 3 

Group 1 
Group 2 
Group 3 

Group 1 
Group 2 
Group 3 

Group 1 
Group 2 
Group 3 



All or 
almost a 
(91% or 
more) 



.s do you think will get this item 






About 


About 


About 


None or 


ill 


three- 


half 


one 


ClJJUOSu 




quarters 




quarter 


none v xuv 




(61-90%) 


( 36-60%) 


(11-35%) 


or less 


70 


30 


0 


0 


u 


40 


20 


35 


5 


0 


0 


10 


20 


40 


30 


90 


10 


0 


0 


A 
U 


20 


50 


30 


0 


0 


0 


5 


25 


50 


20 


50 


50 


0 


0 


u 


10 


30 


40 


20 


0 


0 


10 


15 


20 


55 


30 


60 


10 


0 


u 


0 


15 


50 


35 


0 


0 


0 


0 


35 


65 


75 


20 


5 


0 


A 
U 


30 


25 


35 


10 


0 


0 


5 


15 


55 


25 


65 


30 


5 


0 




15 


25 


40 


20 


0 


0 


10 


5 


35 


50 


70 


25 


5 


0 


A 
U 


20 


50 


5 


25 


0 


0 


10 


15 


40 


35 


50 


45 


5 


0 


A 
U 


15 


30 


30 


25 


A 
U 


0 


0 


15 


30 


• 55 


80 


20 


0 


0 


0 


35 


30 


25 


10 


0 


0 


5 


25 


50 


20 


55 


35 


10 


0 


0 


10 


30 


45 


15 


0 


0 


10 


10 


25 


55 



ERIC 
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Table 7 continued. 



PROFESSIONAL PANEL JUDGEMENTS OF TEST ITEM DIFFICULTY 
FOR THREE HYPOTHETICAL GROUPS OF STUDENTS 



What proportion of students do you think will get this item correct ? 



All or About About About None or 

almost all three- half one almost 

(91% or quarters quarter none (10% 

QUESTION GROUP more) (61-90%) (36-60%) (11-35%) or less) 

















71 


Group 1 


70 


25 


5 


0 


n 
U 




Group 2 


30 


20 


40 


10 


0 




Group 3 


0 


5 


25 


40 


30 


72 


Group 1 


60 


30 


10 


0 


0 




Group 2 


10 


25 


45 


20 


0 




Group 3 


0 


0 


10 


35 


55 


73 


Group 1 


50 


45 


5 


0 


A 

0 




Group 2 


10 


20 


55 


15 


0 




Group 3 


0 


0 


0 


50 


50 


74 


Group 1 


60 


35 


5 


0 


A 

0 




Group 2 


10 


45 


35 


10 


0 




Group 3 


0 


0 


10 


45 


45 


75 


Group 1 


75 


25 


0 


0 


0 




Group 2 


10 


35 


55 


0 


0 




Group 3 


0 


0 


15 


40 


45 


76 


Group 1 


70 


30 


0 


0 


0 




Group 2 


?0 


35 


35 


10 


0 




Group 3 


0 


0 


20 


55 


25 


77 


Group 1 


80 


20 


0 


0 


0 




Group 2 


30 


30 


40 


0 


0 




Group 3 


0 


20 


20 


40 


20 


78 


Group 1 


55 


45 


0 


^ 0 


0 




Group 2 


20 


35 


15 


30 


0 




Group 3 


0 


0 


35 


25 


40 


79 . 


Group 1 


55 


45 


0 


0 


0 




Group 2 


20 


35 


25 


20 


0 




Group 3 


0 


10 


15 


30 


45 


80 


Group 1 


70 


30 


0 


0 


0 




Group 2 


20 


30 


40 


10 


0 




Group 3 


0 


10 


20 


35 


35 



(continued in next page) 



-38- 



Table 7 continued* 

PROFESSIONAL PANEL JUDGEMENTS OF TEST ITEM DIFFICULTY 
FOR THREE HYPOTHETICAL GROUPS OF STUDENTS 



What proportion of students do you think will get this item correct ? 



All or 



About 



About 



About 



None or 







almost all 


three- 


half 


one 


almost 






(91% or 


quarters 




quarter 


none (10% 


QUESTION 


GROUP 


more) 


(61-90%) 


(36-60%) 


(11-35%) 


or less 
—=.——-————-. 


81 


Group 1 


30 


60 


10 


0 


0 




Group 2 


0 


10.5 


52.6 


31.6 


5.3 




Group 3 


0 


5 


5 


10 


80 


82 


Group 1 


20 


65 


10 


5 


0 




Group 2 


5 


5 


55 


25 


10 




Group 3 


0 


5 


0 


20 


75 


83 


Group 1 


60 


40 


0 


0 


0 




Group 2 


5 


40 


40 


15 


0 




Growp 3 


0 


5 


20 


35 


40 


84 


Group 1 


60 


35 


5 


0 


0 




Group 2 


15 


35 


35 


15 


0 




Group 3 


0 


10 


0 


45 


45 


85 


Group 1 


65 


iO 


5 


0 


0 




Group 2 


15 


40 


25 


20 


0 




Group 3 


0 


5 


20 


25 


50 


86 


Group 1 


55 


40 


5 


0 


0 




Group 2 


10 


30 


50 


10 


0 




Group 3 


0 


5 


15 


35 


45 


87 


Group 1 


60 


35 


5 


0 


0 




Group 2 


5 


35 


50 


10 


0 




Group 3 


5 


5 


15 


30 


45 


88 


Group 1 


100 


0 


0 


0 


0 




Group 2 


50 


30 


20 


0 


0 




Group 3 


15 


5 


35 


45 


0 


89 


Group 1 


100 


A 


0 


0 


0 




Group 2 


45 


30 


25 


0 


0 




Group 3 


10 


10 


35 


30 


15 


90 


Group 1 


100 


0 


0 


0 


0 




Group 2 


40 


40 


20 


0 


0 




Group 3 


5 


20 


30 


35 


10 
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Table 7 continued* •• 



) PROFESSIONAL PANEL JUDGEMENTS OF TEST ITEM DIFFICULTY 

r FOR THREE HYPOTHETICAL GROUPS OF STUDENTS 



What proportion of students do you think will get this item correct ? 



All or About About About None or 

almost all three- half one almost 

(91% or quarters quarter none (10% 

QUESTION GROUP more) (61-90%) (36-60%) (11-35%) or less) 



91 Group 1 60 
Group 2 20 
Group 3 5 

92 Group 1 75 
Group 2 15 
Group 3 0 

93 Group 1 45 
Group 2 5 
Group 3 0 

94 Group 1 85 
Group 2 35 
Group 3 5 

95 Group 1 45 
Group 2 0 
Group 3 0 

96 Group 1 65 
Group 2 25 
Group 3 0 

97 Group 1 55 
Group 2 10 
Group 3 0 

98 Group 1 50 
Group 2 10 
Group 3 0 

99 Group 1 35 
Group 2 5 
Group 3 0 

100 Group 1 26.3 

Group 2 0 

Group 3 0 
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Table 8 

NUMBER AND PERCENT OF CORRECT RESPONSES TO EACH 
SCHOOL MASTERY OF READING TEST (SMRT) ITEM FOR 
THREE GROUPS OF STUDENTS DEFINED BY THEIR 
DEGREES OF READING POWER (DRP) TEST SCORES 

(n = 744) 



Group 3 Group 2 Group 1 

\ (low DRP scores) (marginal DRP scores) (high DRP scores) 



SMRT 


(n = 


98) 


(n = 


123) 


(n = 


523) 


ITEM 


Number 


Percent 


Number 


Percent 


Number 


Percent 


1 
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Table 8 continued... 

Group 3 Group 2 Group 1 

(low DRP scores) (marginal DRP scores) (high DRP scores) 
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Table 8 continued* •• . 

Group 3 Group 2 Group 1 

(low DRP scores) (marginal DRP scores) (high DRP scores) 



SNRT 


(n a 


98) 


(n = 


123) 


(n = 


523) 
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Number 
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Number 
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Number 
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SURT SCORES CORRESPONDING TO DRP GROUPS 
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X. THE PPOFESSIOUAL PANEL 



In order to ensure the relevance and usefulness of SMRT-STEPS to 
classroom Instruction and school Improvement, a professional panel was 
established. This panel was comprised of school and Community School District 
office professional educators, as listed on the next page. 

Among other activities, the professional panel v'eviewed the School 
fastery of Reading Test for bias and appropriateness (see Mttachrent # 8), 
provided judgments and opinions related to student performance criteria, and 
recommended ways of improving the usefulness of test results to enhance school 
progress. 

In order to establish the professional panel, each Community 
Superintendent was asked to recomend tvo district office staff. From each 
participatina school, also, two professional staff members including at least 
one fourth grade teacher were invited to join the panel. Specifically, each 
principal was invited to attend. The principal had the option of not 
participating and recommending either an- Assistant Principal or a fourth grade 
teacher Jn his or her stead. In addition to the principal or designee, a 
fourth grode teacher was recommended by the Principal in conjunction with that 
school's United Federation of Teacher (UFT) representative. Both the UFT and 
Council of Supervisor and Administrators (CSA) were apprised of activities 
related to the selection of panel members. 

Curing spring 1986, the Professional Panel met twice at 110 Livingston 
Street. The a<?enda for both the May 8 and June 19 meetings can be found in 
the appendix (see Attachments # 9 and # 10). Remuneration was provided at 
prevailing rates for both teachers (i.e., $21.15 per hour) and supervisors 
(i.e., $23.80 per hour). 
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PROFESSIONAL PAMEL 
Spring, 1S86 



Community School District and School Pedagogical Professionals 
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District Tpst Liaison/Promotional Policv Facilitator 
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David Henrv 


Assistant Princioal 
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14. 


Karen Homier 


Teacher 


15. 


Ronnie Korenge 


Teacher 


16. 


Barbara Levlne 


Teacher 


17. 


Minnifred Mayers 


Teacher 


18. 


Rita Menkes 


Teacher 


19. 


Naomi Killer 


Teacher 


20. 


Vemita Patterson 


Teacher 


21. 


Rhonda Plav/ner 


Teacher 


22. 


Carol Rosen 


Teacher 


23. 


Gil da Tesser 


District Supervisor of Teacher Trainers 


24. 


Leon Weisman 


District Director of Language Arts 


25. 


Bernice Wiley 


Principal 


26. 


Michael Yagoda 


Teacher 



Educational Testing Service Staff 

1. Garlie Forehand Director of Research, Educational Testing Service 

2. riyrtle Rice Research Associate 



Board of Education Headquarters Staff 



1. Richard Guttenberg 

2. Gary Kippel 

3. Charlotte Brown 

4. Ranjit Shivakumar 

5. Lisa Solomon 

6. Ilene Wilets 



Director, Office of Educational Assessment 
Project Director, Office of Educational Assessment 
Executive Assistant 
Computer Systems Consultant 
Educational Consultant 
Psychometric Consultant 



Ex Officio Professional Panel Members 



1. Edna Cohen Principal 

2. Roberta Cohen Principal 

3. Heywood Feierstein Principal 

4. Irwin Grossbard Principal 

5. Maria S. Guasp Principal 

6. Stanley Lavnick Principal 
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X.A: PPCFESSIONAL PANEL OPINIONS ON THE USEFULMESS OF STANDAPDIZED TESTS 



At the May 8th meeting, the Professional Panel was asked to complete the 
Participant Information Form (see Attachment #11). This form consisted of 
four questions in an attempt to gain an overall evaluation of standardized 
tests to use as a guide in the development and implementation of the 
S^?RT-STEPS test. 

Their responses were oroanized into the two categories of supervisors 
and teachers, in order to: 1) reflect areas of common agreement and concern 
between the two groups, and 2) identify the unique concerns of both 
supervisors and teachers. 

The following is question #1 from the Participant Information Form: "In 
general, are standardized reading tests useful to you? If so, in what manner?" 



Common Concerns of Both Supervisors and Teachers ; 

Both supervisors and teachers felt that standardized reading tests are 
useful for funded program evaluation and for identifying the need for 
professional and support staff including consulting services and supplies. 
They also felt that standardized reading tests, are useful in helping to 
identify students in need of small group instruction; in identifying the level 
of the child in relation to grade standing; in identifying promotional gates 
and Limited English Proficiency (LEP) entitled population; and for general 
skills assessment. 

Additional areas of common concern include the opinions that 
standardized reading tests do not reflect overall student perforrrance and that 
they do not always reflect curriculum. In addition, both supervisors and 
teachers want very specific identification of deficit areas so that lesson 
plans can zero in on these areas. 



Unique Concerns of Supervisors : 

Supervisors stressed that standardized reading tests are useful for 
evaluating funded programs; for the comparative monitoring of schools; and for 
assessing accountability of principals. They also felt that these tests were 
valid in determining promotion. The opposite view was expressed in the 
teacher category. Specifically, teachers felt that tests represented too much 
pressure on both teacher and child when used for promotional purposes. 



Unique Concerns of Teachers : 

Teachers placed emphasis on skills assessment for individual instruction and 
stressed that standardized tests provide a fairly realistic assessment of the 
brighter child. For several teachers there was a great deal of concern that 
test results come too late in the year to be of much use as a pedagogical 
guide. 
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« Question number two was: ''In your professional opinion, how can 
standardized reading test results be made more useful?" 



Common Concerns of Both Supervisors and Teachers ; 

Supervisors and teachers felt that test results should be reported 
earlier In the year providing accurate diagnosis of pupil needs toward the 
beginning of the term Instead of at the end. It was felt that this 
specific feedback would serve as a pedagogical guide to Individualize 
Instruction. Both groups also felt that a pre/post format would be desirable; 
that Is to s^y that the same test (or a parallel form) given at the beginning 
of the year Is also given at the end of the year. This way, focus would be on 
areas Indicated as weak at the beginning of the year and Improvement could be 
ascertained easily with the pedagogical approach evaluated at the end of the 
term. 



Unique Concerns of Supervisors ; 

Supervisors felt that standardized reading test results could be more 
useful If; the tests accurately reflected the curriculum; teachers were 
Involved In test preparation; and that tests were field- tested for greater 
reliability and validity* Supervisors also stressed that test results should 
be used as one criteria of many In a child's evaluation and should not be used 
to compare schools and districts serving different populations. 



Unique Concerns of Teachers ; 

Teachers felt that standardized reading test results could be more ' 
useful If the print-out data were easier to read and analyze and If the 
results were more precise In pinpointing areas of difficulty. 
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x.b: professional panel ratings of toe usefulness of 
the school mastery of reading test 



At the June 19 meeting, all Professional Panel members were asked to 
rate their opinions of the potential usefulness of SMRT for several specific 
purposes (see Attachment # 12). As depicted in Table 9, iMst ratings 
reflected the opinion that SMRT would be of "above average" usefulness. In 
this rating scale, "average" is defined as the middle rating category or value 
"4." It is noted, also, that SMRT was Judged as being most useful for 
identifying remedful cases and for planning instruction for both individuals 
and groups. 
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4.S4 


1.39 


0.00 


5.40 


11.10 


44.40 


5.40 


27.80 


5.40 


Cooport clIitH oithin i ichool 


4.44 


1.42 


0.00 


14.70 


5.40 


22.20 


33.30 


!f.70 


5.40 


Coopart Khooli oithin i lyttn 


4.2t 


1.59 


' 0.00 


22.20 


11.10 


14.70 


22.20 


22.20 


5.40 


ttport to partott 


4.28 


1.33 


5.40 


5.40 


11.10 


27.80 


33.30 


14.70 


0.00 


Modify curriculuo 


4.22 


1.58 


11.10 


5.40 


IhlO 


14.70 


33.30 


22.20 


0.00 


Coopart itudMts oith national pttr |roupt 


4.W 


1.47 


0.00 


17.70 


17.70 


11.80 


23.50 


29.40 


0.00 


AfiHs school porfori^ncf 


4.11 


1.49 


5.40 


14.70 


11.10 


5.40 


50.00 


11.10 


0.00 


ScTHt spociil odttcatiooal studtots 


3.71 


1.58 


5.41 


14.70 


27.80 


14.70 


14.70 


11.10 


5.40 


Ntlp tveluatf tiachii^i procodurK or oothods 


3.72 


1.49 


11.10 


22.20 


5.40 


22.20 


27.80 


5.^ 




Rfport to boards of odocation 


3.47 


1.43 


14.70 


11.10 


11.10 


22.20 


27,80 


11.10 




Coapart a systeo nitli systeas across tho country 


3.38 


1.32 


5.90 


23.50 


17.70 


29.40 


II M 

AA.vV 


S 9ft 


Wfi TV 


Htip tvaluatf tiachtr ptrforoanct 


3.28 


1.41 


11.10 


22.20 


22.20 


22.20 


14.70 


5.40 


O.OC 


Holp ttaliatf instructiooil oatorials 


3.22 


1.51 


11.10 


27.80 


22.20 


14.70 


11.10 


11.10 


0.00 


StlKt studtots for kilingua! trainioi 


2.75 


1.20 


11.80 


35.30 


23.5S 


11.80 


17.79 


0.00 


0.00 


Riport to itvspaptrs 


2.38 


l.OS 


23.50 


29.40 


23.50 


17.70 


5.90 


0.00 


0.00 
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X,C: PROFESSIONAL PANEL SUGGESTIONS FOR SCHOOL IMPROVEMEKT PLANS 



At the May 8th meeting, the Professional Panel was asked to complete the Participant 
Information Foo (see Attachment #11). Items three and four of the Participant 
Information Form asked supervisors and teachers for their suggestions for various 
. instructional and organizational strategies, materials, plans and programs likely to 
i- lead to Improved school performance in reading. 

Suggestions Assuming No Additional Resources ; 

Item number three requested suggestions assunlng that: "...current level of resources 
applies... that Is, no additional resources are available." Supervisor and teacher 
Professional Panel aiefliber responses are elaborated helow. 

Cowmen Concerns of Both Supervisors and Teachers ; 

Both teachers and supervisors felt that the following strategies would It^J to Improved 
school performance: smaller registers and small group Instruction; better trained 
teachers and paraprofesslonals; lessons and strategies based on specific needs 
determined by test results. 

Unique Concerns of Supervisors : 

Supervisors emphasized >rore effective use of faculty and grade conferences and more 
teacher input In ordering supplies. 

Unique Concerns of Teachers : 

Teachers focused on high Interest reading materials and basal readers as well as more 
parent Involvement; encouraging parents to foster a positive attitude In their children 
towards books and libraries, their schools and classroom. 

Suggestions Assuming Unlimited Resources 

Item number four asked: "!f unlimited resources were available, what strategies, 
material, plans and/or programs would you recommend? Supervisor and texher 
Professional Panel member responses are elaborated below. 

Common Concerns of Both Supervisors and Teachers : 

Both supervisors and teachers St. o.ngly Indicated smaller class size as an extremely 
desirable goal. If unlimited funds wre available, both groups would have more teacher 
trainers, more resource room teachers and resource people to give demonstration lessons 
In remedial techniques. They would have a paraprofesslonal In every classroom; high 
Interest textbooks; multi-sensory materials tt) help with Instruction; and high quality 
library collections. 

Common Concerns of Supervisors : 

Supervisors stressed classroom library collection^; Individualized reading kits; and a 
rt ource room for teachers. If unlimited funds were available, supervisors also would 
coordinate textbook subject matter to standardized tests. In addition, reading 
management systems correlated to Basal Reading Programs currently In use were suggested. 

Common Concerns of Teachers : 

Teachers suggested the use of computers; first, as part of a reading program using word 

?ame exercises and exercises In syntax; second, to store up-to-date and ongoing records 
or each student; and third, to store test scores and specific test results In a 
diagnostic and prescriptive format. Teachers expressed concern about greater Input for 
^•^^aff into textbook selection, programming and content of teacher training sessions. 

^ ERLC^gg"/^ 67 
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XI. VALIDITY OF THE SCHOOL MASTERY OF READING TEST 

Validity of the School Mastery of Reading Test (SMRT) has 
been established based upon content and concurrent validation 
procedures. Specifically, the relationship between SMRT and New 
York City reading and language arts curriculum was assessed both 
by project staff and by curriculum, language arts and reading 
specialists of the Division Curriculum and Instruction of the 
New York City Board of Education. Furthermore, teacher opinions 
of SMRT were elicited after they administered this test to their 
students. As discussed elsewhere in this report, also, the 
Professional Panel of New York City teachers and supervisors 
provided ratings reflecting their opinions of the usefulness of 
SMRT. In addition, the relationship between SMRT and the 
following other measures of reading is being assessed: the 
Degrees of Reading Power (DRP) test, the Interactive Reading 
Assessment System (IRAS), the Metropolitan Achievement Test, and 
the National Assessment of Educational Progress (NAEP). More 
specific details are provided on the following pages. 
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XI. a: new YORK CITY CURRICULUM AND mESCHOOL MASTERY of READING TEST 



The School Mastery of Reading Test Is related specifically to New York City public 

* school curriculum. In order to estalllsh the congruence between SMRT and New York City 
public school fourth grade reading curriculum^ th^ following three Board of Education of 
New York City (1968» 1969» 1980) publications were ued to define curriculum: Minimum 

. Teachlnq.Essentlals * Grades 3*S > (MrE)> Sequential Levels of Reading Skills (aRS)> and 
the Hanflpooic for Language Arts * Grades 3 and 4 (HLAK in addition » guidance and 
assistance were provided Dy citywioe curriculum specialists of the Division of 
Curriculum and Instruction. 

First, each SMRT Item was reviewed and related to MTE. If It was not possible to 
relate the Item to MTE, then SLRS was used. As a third resort, HLA was reviewed. When 
the Item was located, record was made of both the descriptive category and the grade 
level to which that Item was related. 

Specific examples will Illustrate the manner In which the reading category of each 
test Item was detennlned. Question thirty on page seven of the test booklet reads as 
follows: 

Astronauts visited the moon. They brought back 
rocics. Tftese rocKs are Deing studied . 

30. Vlho visited the moon? 

(A) rocks 

(B) astronauts 

(C) nobody 

Initially, the descriptive categories described In Attachment 1 were used to 
classify test Items. For example, the definition under "Literal Comprehension" reads: 

"The student reads a sentence, several sentences or a short story 
and (1) chooses a sentence that has the same meaning. (2) chooses 
a picture thc:^. best represents the meaning of what was read, and (3) 
answers factual questions about what was read by choosing from a 
list of possible answers. The reading material Includes: simple 
sentences, compound subjects and objects, compound and complex sentences." 

Question thirty on page seven, concerning astronauts visiting the moon, would fall 
under category (3) above: "answers factual questions about what was read by choosing 
from a list of possible answer". 

The reading material would be "simple sentences." 

Next, the Communication Arts-Grade Four section of MTE was examined. In the 
judgment of the professional educators Involved, this test Item was related to "Literal 
Comprehension details may be used to support a point" (see Attachment # 13). 

In fact, MTE further breaks this down by referring to page 26 In the SLRS book (see 
page 26 after: "details may be used to support a point", on Attachment # 13). 
, Subsequently, page 26 In SLRS (see Attachment # 14) "Level C" under "Finding and 
relating details" satys: "Develops the concept that details fill out, extend, make 
specific, exemplify, clarify and document the main Idea." Also, the description under 
^'Language Arts" right below It further clarifies this by stating: "Reads vivid 

* passages, recalls details, and describes or draws simple pictures including details." 
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Another example Is question e1ghty«*e1ght on page twenty«*two of the t6;t booklet. 
The question reads as follows: 

88. Janet put on her bathing suit. She walked 
to tne poou Wnat Happened n"ex^ 

(A) Janet went swimming 
(8) Janet went fishing 
(C) Janet went to school 

As Indicated earlier. Attachment # 1 was used to describe test It^ms. The 
descriptive categories under ""Reasoning Comprehension" reads as follows: ""The student 
reads a sentence, several sentences or a short story and answers Inferential questions 
by choosing fron a list of pictures or written answers/ The reading materials 
Include: ^simple paragraphs, a short story, causal and all or some relationships^ 
predicted outcomes, ccmparlslons and sequencing*" 

Question elghty^elght, concerning what "Janet did after putting on her bathing suit 
and walking to the pool," would fall under "answering Inferential questions by choosing 
from a list of written answers*" The reading material would be "sequencing." 

Next, the "Fourth Grade Communication Arts" section of the MT£*s was reviewed, but 
this type of sequencing question was not listed there* Knotting that fourth grade 
curriculum naturally comprises some of the third grade material, the "Third 
Grade-Communication Arts" section was examined. It was listed there under ••Inferential 
Comprehension" questions* Attachment # 15 Indicates: "Sequential order is a factor In 
predicting outcomes and drawing conclusions*" 

The "page 28" at the end of this sentence refers to the SLRS section on 
"Determining Sequence." Attachment # 16 r*pec1f1es: "Level D: Uses sequential order to 
make Interpretations and predictions". Also, right below under "Language Arts", It 
states: "Predicts outcomes of story stemming from logical sequence of events." 

The purpose In matching the SKRT Items with the New York City Board of Education 
fourth grade curriculum Is to demonstrate ttie extent to which SMRT Is congruent with New 
York City curriculum. All of the SMRT test Items have been analyzed and matched reading 
curriculum In the three curriculum publications mentioned earlier. In other words, SMRT 
has been constructed to reflect curriculum taught In New York City schools. 
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XI. b: teacher opinions Cff TOE SCHOOL MASTERY OF READING TEST 



After administering SMRT, each teacher was asked to complete a one-page survey 
(see Attachment # 5) designed to assess their opinions of the test and testing 
procedures. 

In most Instances, as Indicated In Table teacher opinions were favorable. For 
example, the majority of teachers agreed that: 

The test Items are easy to understand for the students. 

The test questions seem free of technical flaws. 

The test Items are free of bias (for example, sex or ethnic bias). 

The test directions are clear to the test-taker. 

The test administration manual Is easy to use. 

The layout of the test booklet Is convenient for the examinees. 

The reading difficulty levels of test questions are appropriate 
to fourth grade students. 

The majority of tho test questions correspond well to fourth grade 
curricula (for example. Minimum Teaching Essentials). 

The Kiajorlty of teachers Indicated that there Is no need for 
additional practice questions for students. 

At the same time, the majority of teachers Indicated that: 

They somewhat disagree that time limits are adequate, and 

They disagree that the layout of the test booklet Is attractive. 

In other words, there Is some evidence that time guidelines or limits should be 
Improved. In addition, the layout of the test booklet should be made more attractive. 
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Table 10 

TEACHER OPINIONS OF THE SCHOOL MASTERY OP READING TEST 



ttroBfly SoMwhtt SoMwhtt Stronglv 

AftM *9tM AftM DiMgtM Oit«9rM DiiagtM 



r>i« t«8t items »rt «*«y to ' 1 2 3 4-5 6 



|ind«rstand for the students. 



(18.92) (45.9) (18.9) (8.1) (5.4) (2.7) 



Irhe test questions seem free 123456 
fcf technical flaws. (13.92) (38.9) (27.8) (13.9) (5.6) 



phe test items are free of bias 12 3 
ICfor example, sex or ethnic bias). (3^1^) ^.q 



4 



Ifhe testing time limits are 12 3 4 5 6 



■adequate. 



(16.2%) (24.3) (18.9) (27.0) (8.1) (5.4) 



Irhe test directions are clear 1 2 3 4 5 6 

ito the test-taker. 

(24.32) (32.4) (16.2) (21.6) (2.7) (2.7) 

Irhe test administration manual 12 3 4 5 6 

lis easy to use. .(32.42) (48.6) (16.2) (2.7) 



iThere is a need for ad^iitional 1 
practice questions for the student. 



5 6 



(19.42) (11.1) (13.9) (11.1) (38.9) (5.6) 



■The layout of the test booklet 1-2 3 

lis attractive. 



4 5 



. (5.42) (37.8) (32.4) (8.1) (16.2) 

iThe layout of the test booklet 1 2 3 4 5 6 

lis convenient for the examinees. 

(16.22) (40.5) (18.9) (13.5) (8.1) (2.7) 

?he reading difficulty level» of l 2 3 4 5 6 

l^est questions are appropriate to • 

"ourth grade students. ,^ 

^ (8.62) (51.4) (25.7) (11.4) (2.9) 

rhe majority of the test questions 1 2 3 4 5 6 

correspond well to fourth grade 
curriculum (for example. Minimum ,„ 

retching Essentials). (8.32) (61.1) (25.0) (5.6) 
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XI «C: OTHER READING TESTS AND THE SCHOOL MASTERY O F READING TEST 

The initial version of the group administered SMRT was 
validated using the individually administered Interactive Reading 
. Assessment System (IRAS) developed by Robert and Kathryn Calfee. 
The IRAS instrvunent was based on a review of research and 
practice in the teaching of reading. It assesses a set of 
component skills which have generally be.^n accepted as necessary 
for proficient reading. A very strong relationship was found 
between the initial version of SMRT and the IRAS. 

The relationship between the School Mastery of Reading Test 
(SMRT) and both the Degrees of Reading Power (DRP) test and the 
Metropolitan Achievement Test (MAT) was examined. Total test 
scores were available on all three tests for 744 students. 
Correlation coefficients were calculated. The correlation 
between SMRT and DRP was .791 and the correlation between SMRT 
and the MAT was .819. 

The relationship between SMRT and the National Assessment of 
Educational Progress (NAEP) is discussed in detail in another 
section of this report. 
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XII. RELIABILITY OF THE SCHOOL MASTERY OF PEADIHG TEST 



Indices of reliability provide an indication of the extent to which 
a particular measurement is consistent and reproducible (Thorndike & 
Hagen, 1977, p.56). In other words, reliability refers to the necessity 
for dependability in measurement (Kerlinger, 1973, pp.442-443). 
Reliability implies stability, consistency, predictability and accuracy. 
In more technical terms, reliability is the proportion of true variance 
in obtained test scores (see, for explanation, Guilford, 1954, 
pp. 349-354). 

Coefficient alpha is the basic formula for determining the 
reliability based on obtained consistency {Nunnally, 1978, pp. 229-230). 
Coefficient alpha is the expected correlation of cne test with an 
alternative form of the test, of the same length, when the two tests 
purport to measure the same thing (Nunnally, 1967, pp. 196-197). 

In order to obtain reliability estimates for St-'RJ, coefficient alpha 
W4S calculated. Coefficient alpha reliability coefficients are presented 
in Table 11 . These data provide support for the contention that SMRT 
can be usea reliably. 
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lable 11 

Reliability of the School Mastery of Reading Test 

(H»752)*l 

BZSSS«3SSZ«SSSSSS8SZSSSSSSSZSSSXSXSS=S3SS=XSSS8SSSSSSSSSeS8==SSSSSS3 

Aggregate Number CJironbach ' s 

of Test of Alpha 

Items Items 

=SSS=ZZSS=SZS3SSSSS=SaSZZS==S3U==SZSZS3=SZSSSSSSSS3=SS=r.==S=SSSSS==: 

Total Test*2 97 ,9281 

Part One 47 .8611 

Part Two M .8809 

Word Attack Subtest 18 .7510 

Word Meaning Subtest 21 .7880 

Literal Comprehension 31 .8202 

Subtest 

Reasoning Comprehension 27 .7775 

Subtest 



*1 The number of students providing data for these reliability analyses 
v/as 752 as determined by the "listwise deletion" requirements of the 
SPSS-X (1986, p. 872) computer program. Specifically, these analyses 
were comprised of item responses only from test papers where students 
answered all of the 100 questions. 

*2 The three word recognition items at the very beginning of the School 
Mastery of Reading Test were eliminated from these analyses. The 
V/ord Recognition items were used as "sample" items to orient 
students. Total test reliability, therefore, was based upon 97 
rather than 100 items. 
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XIII. PREDICTING DEGREES OF READING POWER TEST SCORES 
FROM SCHOOL MASTERY OF READING TEST SCORES 



It is- likely that the School Mastery of Reading Test (SMRT) 
will be of most usefulness if it is administered at the beginning 
of. the school year. When administered at that time, it can 
provide diagnostic information to assess needs and to help guide 
planning and instruction. To some extent, in addition, SMRT 
administered at the beginning of the school year may be 
predictive of Degrees of Reading Power (DRP) test scores from the 
annual citywide reading testing program conducted near the end of 
the school year. This can be accomplished by generating 
appropriate regression equations. 

For example, a linear regression equation was calculated 
based upon May 1986 test results. The illustrative equation 
expresses the relationship between SMRT and DRP as follows: 

y = .690x - 16.291, 

where "x" represents SMRT scores achieved by students and "y" 
represents predicted DRP scores. This equation could be used to 
predict or estimate fourth grade DRP score from fourth grade SMRT 
scores available before the DRP has been administered. It is 
noted that more sophisticated and more accurate multivariate 
models using SMRT could be developed for predictive purposes. 
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XIV.. NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS 
AND THE SCHOOL MASTERY OF READING TEST 



National Assessment of Educational Progress (naep) has been 
developed to measure how effectively 9-, 13- and in-school 17-year-old 
American students can read (Messick, Beaton, & Lord, 1983). For this 
purpose, nationally representative samples of students within vc\rious 
demographic subgroups are tested (National ;.^sessment of Education 
Progress, 1985). 

NAEP bases each assessment on a wide range of materials and asks 
questions requiring use of a variety of reading skills and strategies. 
Reading selections range from simple sentences expressing a single 
concept to complex articles about specialized topics in science or 
social studies. 

A scale ranging from 0 through 500 has been developed. Both items 
and tests span a wide range of difficulty and are presented in a variety 
of formats. Various points on the scale have been provided criterion- 
referenced interpretations. Items have been calibrated using the three- 
parameter latent trait model (See, for discussion; Messick, Beaton, & 
Lord, 1983, pp. 43-55). 

Items are reviewed for potential bias before being accepted by NAEP 
for administration. Specifically, NAEP items are reviewed by educators 
on the basis of their academic appropriateness, effectiveness, freedom 
from bias or stereotyping, and sensitivity to racial, ethnic, religious 
and political groups. After test administration, item response curves 
are analyzed for potential bias. 

The relationship between SMRT and NAEP is being determined. In 
effect, the current study is designed to improve local school level 
diagnosis and prescriptions for progress by using NAEP items and norr.^' 
(Seo footnote #1). The primary intent is to determine the feasability 
of: 

1) obtaining norm- referenced interpretations of SMRT results with 
respect to NAEP national norms, 

2) demonstrating the extent to which SMRT results relate to NAEP 
absolute performance, 

3) establishing a cost-effective source of items by incorporating 
NAEP items within SMRT, and 

4) estimating costs and resources required. 



1 

In addition, it is noted, that a somewhat different potential role 
for NAEP in assisting the development and implementation of local 
educational standards has been defined by Messick (1955). 
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In order to achieve these objectives, NAEP Items were evaluated 
with regard to Item content, format and general appropriateness for 
fourth grade students. Jt was determined, consequently, that some NAEP 
Items could be Incorporated within SMRT. This decision was based upon 
the fact that current elementary school level NAEP items were designed 
for grade three students and have sufficient range for grade four 
students. In the recent past, elementary school level NAEP items were 
designed for grade four studentci. It is noted that SMRT is designed for 
relatively low achieving fourth grade students. Furthermore, SMRT is 
most likely to be administered early in the school year for maximum 
diagnostic usefulness. 

Scxne NAEP items ara so similar in foi.mat and content to some SMRT 
items that, if mixed together, it would be difficult to determine the 
source of each. At the ssme time, some SMRT item types are not matched 
by NAEP items. 

A more specific assessment of the congruence between NAEP and SMRT 
requires detailed review of item content, foicmat and psychometric 
characteristics of items from both sources. In addition, analyses of 
scales and assessment cf the dimensionality of both NAEP and SMRT are 
required. Such reviews are being conducted. 

At our request, the Educational Testing Service (ETS) obtained 
permission for use of NAEP items within SMRT. Permission was granted to 
use NAEP items under "reasonable constraints". Specifically, it is 
understood that: 1) NAEP items will not be p7l)lished or inappropriately 
disseminated, 2) NAEP items will not be used for pre-test practice or 
instruction, and 3) appropriate steps will be taken to insure adequate 
security of NAEP items. 

Selection of particular items were based upon item scale value, 
content and format. A total of 16 NAEP items were selected for testing. 
These NAEP items were embedded within both Parts I and II of the two- 
part SMRT test. 

In addition to obtaining traditional item statistics, latent trait 
analyses are being conducted. The intent is to use the NAEP items to 
determine the SMRT item parameters. First, it must be demonstrated that 
the test data fits the Latent Trait Model (See, for discussion; 
Hambleton £ Cook, 1977? Wright & Stone, 1979). Subsequently, SMRT item 
calibrations will be calculated. In effect^ the overall goal is to 
establish a common SMRT-NAEP scale with a Latent Trait calibrated item 
pool. Consequently, new forms and levels of SMRT can be d«signed which 
will be based upon New York City curriculum and will yield NAEP norm- 
referenced information. It is noted that a similar strategy (See, for 
discussion; Lenke, Oswald, & Kippel, 1982) was followed in the 
development and administration of the customized New York City 
mathematics test for five years from Spring 1981 through Spring 1985. 
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XV. DEVELOPMENT OF SUBTESTS FOR THE SCHOOL MASTERY OF READING TEST 



Test Items were categorized by subtest based upon the professional opinions of 
several curriculum, reading, research and teaching special Ists. Subtests were developed 
using the definitions provided In Attachment # K 

The School Mastery of Reading Test (SMRT) Is comprised of two parallel sections 
^(l.e.. Parts I and II), each containing 50 Items. The Items within each part serve to 
measure a particular .facet of reading ability. The five following types of Items were 
Included In the Miy, 1966 test administration: 1) word recognition, 2) word attack, 3) 
^word meaning, 4) literal comprehension, and 5) reasoning comprehension. 

As Indicated earlier, the numbers of Items within the five categories were: word 
recognition— 3 Items, word attack— 18 Items, word meaning— 21 Items, literal 
comprehension— 31 Items, and reasoning comprehension— 27 Items. 

In order to validate and confirm the placement of Items within the particular 
subtests^ the use of factor analytic statistical techniques was considered. 
Consequeritly, we conferred with an expert In such techniques. Dr. Phil Merr1f1eldj» who 
Is a professor and measurement speclaHst at New York University. Dr. Merri field 
recommended an alternate statistical approach. 

Factor analytic procedures do not appear to be appropriate for the development and 
confirmation of subtests on mastery tests such as SMRT. The factor analytic technique 
relies on the assumption that test scores are normally distributed. I.e. some students 
did very well, some did poorly, and the majority fell somewhere In-between the two 
extremes. The SMRT, however. Is a test measuring fourth grade reading mastery, which 
was administered to fourth graders est the end of their r.;:adem1c year. As a result, most 
students obtained relatively high scores because they had mastered fourth grade reading 
skills. As expected, consequently, 'che distribution of test scores was "^negatively 
skewed" rather than normally Ulstirlbuted. This departure from the blvarlate normal 
distribution might confound any results obtained through factor analytic methods 

Or. Merrlfleld recommended that the subtests be validated by rank«order1ng the 
Items In each of Parts I and II within subtests based upon Item difficulty. For 
example, the Literal Comprehension subtest Items In Part I would be rank-ordered based 
upon Item difficulty. Secondly, the additional Literal Comprehension subtest Items In 
Part II would be rank-ordered separately frt^ Part I Literal Comprehension Items. 
Subsequently, the same procedure would be followed with Items In Parts I and II for each 
of the other subtests. 

After Items are matched on Item difficulty, correlations between Parts I and II 
for each subtest must be calculated. For example, to what extent the Literal 
Comprehension items In Part I of the SMRT correlate with those In Part II? A high 
correlation suggests the Items are all measuring a common reading skill. The group of 
Items may then be considered a cluster or factor, representative of one of the various 
dimensions comprising reading ability. This Is similar. In effect, to estebllshing two 
alternate or parallel forms of the Literal Comprehension subtest* 

In addition, correlations between different subtests for each of Parts I and II of 
SMRT should be obtained. For example, what Is the correlation or relationship betwen 
Literal Comprehension and Reasoning Comprehension within Part I? In this case, low 
correlations would su<;gest distinct subtests. In contrast, high correlations might 
Indicate that certain subtests might be combined. This type of Information would be 

* extremely useful In the test validation process. 

These particular statistical techniques may be thought of In terms of split-half 

* reliability methods. In effect, the underlying structure of the test Is being examined 
by determining the extent to which the Items relate to each other. In other words, are 
the Items measuring the same or different facets of reading ability? How strong Is this 
relationship? How weak? All these questions are Important In terms of the validation 

5 SMRT subtests. 
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The relationship between the four primary subtests is presented in 
Table 12 » Review of the correlations reveals that the highest 
correlation (1»e», J6^) Is between the Literal Comprehension and 
Reasoning Comprehension subtests* The lowest correlation is between Word 
Attack and Reasoning Comprehension* 

Table 12 



Intercorrelation Matrix of School Mastery of Reading Test Subtests 

(N « 889) 



SUBTEST 


WORD 
ATTACK 


WORD 
MEANING 


LIiERAL 
COMPREHENSION 


REAS()NING 
C0l?REHENSI0N 


WORD ATTACK 


1.000 


.670 


.644 


.628 


WORD MEANING 




1.000 


.734 


.698 


LITERAL 
COMPREHENSION 






1.000 


.764 


REASONING 
COf!PREHENSION 








1.000 
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XVI. Potential Short- and long-Term Objectives 



Short-Tenn Objectives 

.Office space must be acquired and computer terninal hardware and software irust be set-up 
and available for use. 

budget must be established for the 1986-1987 school year. 

Consultants must be acquired for the 1986-1987 school year. 

In order to obtain necessary funds i / all aspects of S^^RT, a technically sophisticated 
research proposal must be developed for submission to the Office of Educational Research 
of the Department c/f Education, and possibly, to various foundations. 

Latent trait analyses to determine the feasibility of determining common scaling between 
Sf'^RT and National Assessment of Educational Progress must be conducted. A common scale 
between SMRT and HAEP will enable: cost-effective source of items, comparisons with 
National HAEP results, and reference to NAEP absolute performance. 

School improvement plans of participating school must be reviewed. 

SMRT and the test administration manual must be reviewed and revised based upon the fay 
1986 test administration. 

Time guidelines for administration SWRT will be reviev/ed and the possibility of 
administering SflRT as a "powe'r** rather than as a "speed" test will be determined. 

The graphics, layout, format and the quality of both the SMRT booklet and the test 
administration manual will be improved. 

Implications of SMRT for the Chancellor's Minimum Standards Program must be determined. 

SMRT may be administered to more schools in both October and May to develop local non^s 
and/or expectations of performance and to further elaborate standardization and 
technical test characteristics. 

Item analyses will be performed and psychometric characteristics of SMRT will be 
established. For example, the foUov/ing may be included for items, subtests and total 
test, as appropriate: difficulty levels, distractor analysis, central tendency, 
variability, reliability, item intercorrelation, item-subtest-total test correUtion, 
and proportion of responses achieving mastery or manifesting non-mastery. 

A computerized S^•.RT item bank will be established. Procedures fo^ writing or acquiring 
new test items required for different forms and levels of SfIRT must be established. 

Descriptive characteristics of participating students, staff and school b will be 
obtained and summarized from New York City Board of Education School Profiles reports. 
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Long-Tem Objectives 

Maiitery criteria must be devt^^wped for SMRT items, subtests and/or total test score. 

A SMRT student answer sheet must be developed and appropriate scoring software must be 
developed* 

Prototypes of meaningful reports for school aclninistrators and teachers must be 
' developed. 

Relevance of SMRT for special education and limited English-proficient students must be 
determined. 

Demonstration of the relationships between SMRT, the Degrees of Reading Power (DRP) test 
and the Metropolitan Achievement Test (MAT) must be conducted. 

Innovative uses of computer technology for test delivery, scoring and interpretation 
will be explored. 

A new SHRT answer key must be developed to establish standardized response categories. 

SHRT score profiles must be related to prescriptive choices of instructional programs, 
strategies and materials for local school imprcvement. 

SfIRT should be extended to either grade three or grade seven students. 

8otli Fall and Spring Citywide norms should be developed. 

The relevance of SKRT to the Promotional Gates Program must be demonstrated. Sf!RT ray 
assist in individual tudent diagnosis, prescription and Promotional Gates "appeals" 
decision-making. 

The relationship between SMRT and Comprehensive Assessnent Report results and 
conclusions must be determined. In particular, the relationship with Degrees of Reading 
Power test results must be determined. 
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Description of Schoo l Mastery of Reading Test Category/Subtest 



Cateqory/Subtest 



Description 



I. Word recognition 


(2) read, a word and chooses a matching picture, and (3) looks at a picture 
and chooses a matching word. The foUowing words are included: of, waJ cat 

pxano, birdcage, castle, swords. 


2. Word attack 


I5!t"o?'*!o^H»^}-*'""-* ""'^ and chooses a word with the same sound from a 
list of words (u, a, o, oi» ow, f, ch, t, gh), and (2) reads a word with a 

und"rUnS'"i T f^^.l 'T * '''' " "^''^ "^'^ sa«e "o J Is t,e 


3. Word neaning 


!!!^«"f"J*;;Irdi^ '° definitions, (2) chooses synonyms and anto- 

f^lotlL^lrd. t t J r"**" "P^^^*" ^" sentences. The 
foUoHing words are included: ring, cry, chair, night, above, glad, slow, sick 

e":io":i.\1;n?^';k:5*^'- -"^e,\et;ii, i;s::^'' 


4. Literal compre^ 

hsriBion 

%%VP vs4 


r^ente^ce't^afSa: Jh"'*""''' sentences. „r a short story and (1) chooses 
a sentence that has the same meaning, (2) chooses a picture that best represents 
the meaning of what was read, and (3) answers factual questions ab"t Z^ was 
luLl^. r'f"^ ^"^ ' The reading aateri^l In- 
sentences! compound subjects and objects,, compound and complex 


5. Reasoning compre- 
hension 


SJ^om!!? sentence, several sentences, or a short story and answers 

ih! ^^T^""^ ^ °^ " answers. 
I??/»n;l uJ" '^"8^* paragraphs, a short story, causal and 
aii/some relationships, predicted outcomes, cxxrparisons emd sequsncing. 


6. Comprehension: 
cloze 


wcrds'm^^^Lr^f " ""S ^'°r^*' °' paragraphs each) with seven 

off? V u ^'^ "^"^"8 ""dent chooses from a 
list of five words the word that best compleres the meaning of the story. 
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QUANTITY AND IDENTIFICATION NUMBERS OF TESTS TEST ADMINISTRATION MANUALS 



Potential 

Number of Number Range of Range of 

3chool Students on 25% of Tests of Tests Admin. Manuals 

Number Register Overage Required Numbers Numbers 



CSD #17 191 




134 






34 




168 


1 




168 


1 


- 10 


289 




157 






39 




196 


159 




364* 


11 


- 20 


398 




197 






49 




246 


365 




610"* 


21 


- 30 


(Subtotal) 














(610) 










(30) 


CSD #19 213 




129 






32 




161 


611 


- 


771 


31 


- 40 


290 




134 






34 




168 


772 




939 


41 


- 50 


328 




82 






21 




103 


940 


• 


1042 


51 


- 60 


(Subtotal) 














(432) 










(30) 


CSD #21 90 




76 






19 




ii5 


1043 




1137 


61 


- 70 


212 




89 






22 




111 


1138 




1243 


71 


- 80 


3i9 




67 






17 




84 


1249 




1332 


81 


- 90 


(Subtotal) 














(290) 










(30) 


Total 














(1332) 










(90) 


* Box 1 of 2, 
** Box 1 of 2, 


# 
# 


s 169-266; 
s 365-488; 


Box 
Box 


1 rvi rvi 


of 
of 


2, 


#'s 267- 
#'s 489- 


364. 
510. 











88 



f 





NAME (Last. First. M.I. 



® ® ® ® 

® ® ® 0 
©©©© 
© © © © 
© ©©© 
©©©© 
©©©© 
© ® ® ® 
©©©© 

©00© 
©® ©© 

©©©© 
© ® © ® 
© © © © 

© © © © 

©©0© 
®® © © 
©©©© 
©© ©© 

©©©© 
© © © © 

©©©© 

© ® ® © 

© © © © 

©©©© 

©C ®© 



© © © 
©©© 
©© © 
® ® © 
©®© 
©©© 
©©© 
©® © 

0©© 
©0© 
©©© 

©©© 
® ® ® 
©©© 
© © © 
©©© 
® © © 
© © © 
©® © 
©©© 
® © © 
©© © 
© ® © 
©©© 
©©© 
©0© 



ocrocra 
© © © © © 

©©©©© 
©©©©© 
© © © © © 
© © © © © 
©©©©© 
© © ©© © 
© © ® © © 

© © © © © 
©©©©© 
©©©©© 

©©©©© 
® ® ® ® ® 
® ® ® ® ® 
© © © © © 

©©©©© 
©©©©© 
©©©©0 

©© ©© 0 

0©©©© 
0® ®® ® 
© © © © © 

® © ® ® ® 
© © © ® © 

© © © © © 

0©©©® 



©0© 
©0® 
©© © 
© © © 
©©© 
©©© 
©©© 
©® © 
©©© 

©©0 
©©© 

0©© 
® ® ® 
® ® ® 
© ® © 
©©© 
® ® ® 
® ® ® 
©©© 
©©© 
© © © 
© © © 
© ® © 
© © © 
© © © 
©0© 



O 

©©©©© 
© © © © © 
©©©0© 
© © © © © 
© © © © © 
©©©© © 
©©©©© 
© © © © © 
000©© 

© © © © © 
© ©©©© 
©©©©© 
® ® ® ® ® 
® ® ® ® ® 
© © © © © 
©©©©© 
© © © © © 
© © © © © 
© © © © © 

©00©© 
© © © © © 

© ©© © © 
© ®® © © 
0 © © © © 

© © © © © 
©©0©® 



GENERAL PURPOSE - NCS - ANSWER SHEET 

FOR USE WITH ALL NCS SENTRY OPTICAL MARK READING SYSTEMS 
SEE IMPORTANT MARKING INSTRUCTIONS ON SIDE 2 



BIRTH DATE 


IDENTIFICATION NUMBER 


SPECIAL CODES 


MO. 


DAY 


YR. 


A 


B 


C 


D 


E 


F 


G 


H 


1 


J 


K 


L 


M 


N 


0 


P 


M«0 

A*.© 

J""© 
Jul© 

»"«© 

s.pO 

OCI© 
""»© 

0-0 






































©© 
©© 
©© 
©© 
© 
© 
© 
© 
© 
© 


© © 
©© 
©© 
©® 
©© 
© © 

©© 

©0 
©© 
©© 


©®®®®®©®©© 
©©©©©©©©©© 
®®®®©©©©©© 
©©©©©©©©©© 
©©©©©©©©©© 
©©©©©©©©©© 
©©©©©©©©©© 
©©©©©©©0©© 
©0©©©©©©©© 
©©©©©©©©©© 


©©©©©© 
©©©©©© 
©©©©©© 
©©©©©© 
©©©©©© 
©©©©©© 
©©©©©© 
©0©©©© 
©©©©©© 
©©©©©© 



A B C D E 
10®®©® 

A B C D E 
2©©®®© 

A B C D E 
3CD®®0® 

A B C D E 
40©©©© 

A B C D E 

S©©©©® 

A C C D E 
8©©®©© 

■A B C D E 
7©©©©© 

A B C D E 
• 0©©©© 

A B C D E 

9©©©©© 

A B C D E 
lOCD®®©© 



A B C D E 
11©©®®® 

A B C D E 
12©©©©© 

A B C D E 

13 CD©®©© 

A B C D E 

uCD®®®® 

A B C D E 
IS©©©©® 

A B C D E 

leCD®®®® 

A B C D E 
170©©©© 

A B C D E 
luCD®®©© 

A B C D E 
19©©©©© 

A B C D E 

20 CD©®©© 



A B C D E 
51©©©©© 

A B C D E 

52©©©©© 

A B C D E 
53©©®©© 

A B C D E 
54©©®©© 

A B C D E 
55©®®©® 

A B C D E 
56 CD®®®® 

A B C D E 
57©©©©© 

A B C D E 
58©©©©© 

A B C D E 
59©©©©© 

A B C D E 
60©©©©© 



A B C D E 
81 ©0©©© 

A B C D E 
62©©©©© 

A B C D F. 
63 CD®®©® 

A B C D E 
64©©©©© 

A B C D E 
65©©00© 

A B J E 
66©©@©(D 

A B C D E 
670000© 

A B C D E 
68©®©©(D 

ABODE 
68©®©©® 

A B C D E 
70©®©©© 



A B C D E 

21 CD®®®® 

A B C D E 
22CD®(i>®® 

A B C D E 
23©©©©© 

A B C 0 E 
2<©(D©©CD 

A B C D E 
25©©©©© 

A 3 C D E 
26000©© 

A B C D E 
270©©©© 

A B C D E 
28©©©©CD 

A B C D E 
29©©©©© 

A B C D E 
30©©©©© 



A B C D E 
31©©©©© 

A B C D E 
32©©©©© 

A B C D E 
33©©©©© 

A B C D E 
34©©©©(D 

A B C D E 
36 CD®®©© 

A B C D E 
36©©©©© 

A B C D E 
37CD®(I)©© 

A B C D E 
38©©©©© 

A B C D E 
39©©©©© 

A B C D E 

40 CD®®©© 



A B C D E 
41©©©©© 

A B C D E 
42 CD®©©© 

ABODE 
43CD®Ct®® 

A B C D E 
44©©©0© 

ABODE 
45©(D©©® 

A B C D E 
46©©©©© 

A B C D E 
47©©©©© 

A B C D E 

4aCD©(l)©© 

A B C D E 
490©©©© 

A B C 0 E 
5C©<D©0(i) I 



A B C D E 

71 CD®©©© 

A B C D E 

72©©©©© 

A B C D E 
73©®®©® 

A B C D E 
74CD®(I)®® 

A B C U E 
75©©©©© 

A B C D E 
76©©©©© 

A B C D E 
77©©0©© 

A B C D E 
78©©©®® 

A B C D E 
79©©©©© 

ABODE 
80©©©©® 



A B C D E 

81 ©©@©© 

A B C D E 

82 CD®®©© 

A B C D E 
83©©©©© 

A B C D £ 
84©©®©(I) 

A B C D E 
85©®©©® 

A B C D E 
86©®(3)©® 

A B C D E 
87©©©©© 

A B C D E 
88©©©©© 

A C- C D E 
89©©©©© 

A B C '<) E 
90©©©®© 



A B C D E 
9lCD®(l)©© 

A B C D E 
92®®®®® 

A B C D E 
93 CD®®®© 

A B C D E 
94O®®0® 

A B C D E <\ 
95®®®®©^ 

A B C D E ^ 
96©©©©©* 

A B C D E S 
97CD®©©© ¥ 

A B C D E ^ 
98®®®®® k 

A B C D E >C 

99CD®®®® 
A B C D E 
100 CD®®®® 



1 1 I I I I I I i I e I I e I I I I I I I I I 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 i 1 1 1 null 



ERIC 



90 



5 /4^T7hCHHft**T^^ 

\' We art inttresttd in dtveloping and Improving upon the School Mastery 

of Reading Test (SMRT) . Vour expertise as a teaching professional would 
aid us in this endeavor. Below are several statements concerning various 
aspects of the test. Please indicate the extent to which you either 
"agree" or "disagree" with EACH statement by circling the number corresponding 
to your opinion. Please DO NOT SIGN YOUR NAME on the questionnaire as all 

' ♦ responr^es are to. remain anonymous. Remember* there are no right or wrong 
opinions. 



AgtM A«tM A«tM OitagrM DisaftM DiMftM 



the test items are easy to 
^under stand for the students. 



The test questions seem free 
of technical flaws. 



the test items are free of bias 
(for example, sex or ethnic bias) 



Th« testing time limits are 
adequate* 



T^e test directions are clear 
to the test- taker. 



The test administration manual 1 
is easy to use. 

There Ij a need for additional 1 
practice questions for the student. 



The layout of the test booklet 
is attractive. 



The layout of the test booklet 
is convenient for th« examinees. 



The reading diffdcul'!:y levels of 1 
test questions are appropriate to 
fourth grade students. 



The majority of the test questions 1 
correspond well to fourth grade 
curriculum (for example, Minimum 
(j'lng Essentials). 



Types of Test Scores 



In your opinion, how useful are each of these types of scores? 
a. ftaw scores 

J 1 J I 1 I t 



Not at all 
useful 

b. Normal curve equivalents 
1 I I L 



Extremely 
useful 



Kot at all 
useful 

c. Percentile rankings 

J L i_ 



Extremely 
useful 



Not at all 
useful 



Stanines 
I 



Extremely 
useful 



Not at all 
useful 



Extremely 
useful 



e. Grade Equivalents 
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useful 
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Extremely 
useful 
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Types of Test Scores JfPhtH^i^^'^ 

Raw Score . 

The first quantitative result obtained In scoring a test. Usually the number 
of right answers; number right minus some fraction of number wrong; time 
required for perfomance; number of errors, or similar direct/ unconverted, 
^ uninterpreted measure, (see source # 1 below) 



Normal Curve Equivalent jWCE) . 

This score, used primarily for reporting In federally funded programs such as 
Title I, has a mean of 50 and a standard deviation of 21.06. (see source # 2 
below) 



Percei^tlle . 

A point (score) In a distribution at or below which fall the percent of cases 
Indicated by the percentile. Thus a score coinciding with the 35th percentile 
(P35) Is regarded as equaling or surpassing that of 35 percent of the persons 
In the group, and such that 65 percent of the performances exceed this score. 
"Percentile'* has nothing to do with the percent of correct answers an examinee 
makes on a test, (see source # 1 below) 



Stanlne . 

One of the steps in a nine-point scale of standard scores. The stanine (short 
for standard-nine) scale has values from 1 to 9, with a mean of 5 and a 
standard deviation of 2. Each stanine (except 1 and 9) is 1/2 standard 
deviation in width, with the middle (average) stdnine of 5 exte3nd1ng from 1/4 
S.D. below to 1/4 S.O. above the mean, (see source # 1 below) 



Grade Equivalent (GE) . 

The grade level for which a given score is the real or estimated average. 
Grade-equivalent interpretation expresses obtained scores in terms of grade 
and month of grade, assuming a ^0-month school year (e.g.» 5.7). Since such 
tests are usually standardized at only one (or two) point(s) within each 
grade, grade equivalents between points for which there are data-based scores 
must be ''estimated'' by interpolation, (see source # 1 below^ 



Sources: 1. The ^sych^-logical Corporation (Undated). A Glossary of 

Measurement Terms . (Test Service Notebook 13 ). san Anton 1 o , 
Texas. 

2. Seashore, H.6. (1980) Methods of Expressing Test Scores . (Test 
Service Notebook 148 ) . San Antonio, Texas: The Psychol ogicaT" 
torporatfon. 
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MAMS: 



DISTRICT: 



ITEM BIAS AND COMMEHT SHEET /^C^>€e»J7^^ fy 

The Itetns in thli test were developed to be as unbiased as 
possible with respect Co sex. race, ethnicity, and urban/rural origins. 
In the event that we have Inadvertently .included ite«s that might dis- 
criminate against »o9c" group or groups e£ students, we would like you 
to indicate by item nuuSber any item that you feel is basically unfair. 
For each Item so indicated, please explain the source of the bias that • 
you feel exists. Under "Other Comments", add any further reaction you 
may have to the ieea(s). 

Item #*s Source of Bias Other Cogments 



. ERJC 



(Use other side for tddltlontl coonents) 



SMRT-STEPS 



School Mastery of Reading Test 
System To Enhance Progress of Schools 



PRELIMICNARY PROFESSIONAL PANEL MEETING 

May 8, 1986 
4:00 p.m. to 8:45 p.m. 



AGENDA 



Registration and Distribution of Folders 



Welcome, Introductions and 

Brief Overview of Current Activities 



Gary M. Kippel 



Greetings 



Richard Guttenberg 



Overview of Proposed Long-Term Activities 



Garlie A. Forehand 
Myrtle Rice 



Test Administration 



Ilene Wiiets 
Charlotte Brown 



Self-Scoring of Answer Sheets 



Panel Members 



Discussion of SMRT Test and 
Adiittinistration Procedures 



All P^xticipants 



Written Reactions Provided on 
"Item Bias and Comments" Sheets 



Panel Members 



Discussion of the Usefulness of Test 
Information to Improve Schools 



All Participants 



Written Responses to Questions 

on Improving Diagnosis and Prescription 



Panel Members 



Summary, Plans for Next Meeting and 
Conclusion 



S6 



Garlie A. Forehand 
Gary M. Kippel 



N*w York City 
BoAid of Education 



no Livingiton Strett 
Brooklyn. New York 11201 



Ch»l Ettcubv* ior Iniiructon 



H«lh«n OutnoriM 
Chanetlior 



Oiiic* ei Idueaboul Amm*!! 
Itich«rd SuHtnbtrq 
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SMRT-STEPS 

School Masteiy of Reading Test 
Systew To Enhance Progress of Schools 

PRELIMINARY PROFESSIONAL PANEL MEETING 

June 19, 1986 
4:00 p.m. to 8:45 p.m. 

AGENDA 

Registration and Distribution of Folders 

Usefulness of Standardized Reading Tests 

Relationship Between School Maste?y of Reading Test 
and New York City Curriculum 



Improving the School Mastery, of Reading Test 
and Test Adnini strati on Procedures 



Panel Judgements of Mastery Criteria 
Preliminary Field Test Results and Future Analyses 
Enhancement of School Progress Efforts 
Test Scores and Reports 
Plans for the Future 

EVALUATION • USD • TESTING • DATA ANALYSIS 
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PARTICIPANT INFORMATION FORM 

SPRING, 1986 PRELIMINARY PROFESSIONAL PANEL 
Improving Diagnosis and Prgscription 



May 8, 1986 
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SPRING, 1986 PRELIMINARY PROFESSIONAL PANEL 

Hay 8, 1986 
USEFULNESS OF TEST RESULTS FOR INSTRUCTION 

In general, are standardized reading test results useful to you? 
(Check one box) 

[II YES [II NO 

If yes, in what manner? Please specify how these test results 
are useful to you? 



ERIC 
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SPRING, 1986 PRELIMINARY PROFESSIONAL PANEL 

May 8, 1986 
USEFULNESS OF TEST RESULTS FOR INSTRUCTION 



In your professional opinion, how can standardized reading test 
results be made more useful? 
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SPRING, 1986 PRELIMINARY PROFESSIONAL PANEL 

May 8, 1986 

PANEL SUGGESTIONS FOR PRESCRIBED SCHOOL ACTIVITIES 



Effective use of diagnostic information to improve school 
progress requires the implementation of specific activities in 
response to test scores. For example, when reporting reading test 
results, it may be desirable to provide school supervisors and 
teachers with a brief description or computeri26;d "menu" of various 
instructional and organizational strategies, materials, plans and 
programs which are likely to lead to improved school performance in 
reading . 

Inasmuch as you are exemplary professional educators, we would 
appreciate your suggestions of such strategies, materials, plans 
and/or programs. Please assume that your current level of resources 
applies. . .that is, no additional resources are available. 
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SPRING, 1986 PRELIMINARY PROFESSIONAL PANEL 

May. 8, 1986 

PANEL SUGGESTIONS FOR PRESCRIBED SCHOOL ACTIVITIES 



If unlimited resources were available, what strategies, 
materials, plans and/or progreuns would you recommend? 



o 102 



For each objective, please circle the number which best expresses your opinion 
of the degree of usefulness of the School Mastery of leading Test. 



NOT 

AT 

ALL 

USEFUL 



Measure educational status of Individuals 

Measure educational "growth" of Individuals 

Identify remedial cases 

Select students for bilingual training 

Screen special education students 

Help plan Instruction for individuals 

Help plan Instruction for class groups 

Detect system-wide general strengths/weaknesses... 

Help evaluate teaching procedures or methods 

Help evaluate Instructional materials 

Help evaluate teacher performance ' 

Compare students with national peer groups 

Compare classes within a school 

Compare Schools within a system 

Compare a system with systems across the country.. 

Assess school performance 

Provide Information for state funding 

Determine minimum competency standards 

Modify curriculum 

Report to newspapers 

Report to boards of education 

1^<iport to parents 

Report progress to students 



2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 



3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
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US 



4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 



5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 
5 



6 
6 
6 
6 
6 
6 
6 
6 
6 
6 
6 
6 
6 
6 
6 
6 
6 
6 
6 
6 
6 
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COMTENT AREA AND TOPIC 



SKILLS AND KNOWLEDGE 



Exposition 



Factual materials give useful knowledge ahout the world. 



PROCESSES 

Following skills generally 
applicable to content areas 
as well as to literature 
Comprehension 

titeralT^, 



Inferential 



Critical 



Word Meaning 
Vocabulary Oevelopnent 

Contextual Clues 

Structural Analysis 

Phonic Ax?aly5is 
Should have been com* 
pleted by this grade 
except for individual 
students who need 
reinforcement 
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Details way be used to support a point. **P- 



26 



Details nay be inferred from the context. **P* 31 
Characters can be analyzed by their actions. **P* 30 
Outcomes may be predicted based upon st&ted fact. **P* 30 

Personal experiences may be related to a narrative or 

expository selection. **P* 34 
Fact and opinion may be distinguished based upon clue iiords 

such as> In my opinion and I believe that , **P- 34 



Vocabulary also expands through including content area words ^9 
Words evoke <^motional and sensory responses. **P« ^ 

Word meaning may be found contextual ly through comparison 
and/or contrast, ♦♦p. 22 

Words of Qore than one syllable can be divided betN^een the 
syllables, ♦♦p. 20 

(For detailed information on the various phonemes see the 
tables appended to this bulletin.) 



105 



I 



sKia 



LEVEL B 



1 

LEVEL C 



LEVEL Ty 



Develops the understanding that a 
series of connected sentences (spo- 
ken or written on this level of ma* 
turity and understanding) contain 
details i%hich are related to each 
other and to the main thought* 



Ungvafc Alts: Learns to find speciHc^ 
details in simple reading nutertal by And* 
ing answers to questions, e.g.. What dk! 
the children bring to the party? 

Sdenee: Picks out, from an experience 
chart, details related to the main idea, 
e^., Whit were tome of the things our 
magnet could pkk up? 

Social StiHties: Reads simple text for an- 
twere to questions, e^.. In what way does 
the policeman help us? 



Develops the concept that details^ 
fill out, extend, make specific, ex- 
emplify, clarify, and document the 
main idea, 



Language Aitt: Reads vivid pasuges> 
recalls details ^ and describes or draws 
simple pictures including details* 

Scfencc: Reads and follows detailed step- 
by-step directions in order to perform a 
simple experiment, e.g.. 

Plants Need Water 
Materials: 2 four-inch pots 
2 similar plants 
soil 
watfr 

Directions : 

1. - Place soil in each pot. 

2. Place plants in soil. 

3. Every day, water one pot only. 
Question: What did the plants look like 

on the first day? 
after a week? 
after 2 weeks? 



Develops the ability to use details 
in preparing material for oral or 
written prcKntation: develops the 
ability to UK details in making a 

point. 



I LMgi^ge Afftt: Answers questions (taken 
from material read) asking for details- 
who, what, where, when, how big, what 

color. 

Sckttot: Uses detuls from printed mat*- 
riab to prove or refute a point, e^., wheels 
nKJYf things more easUy. 

Social Studies: Supplies details from 
material read under a stated main idea* 
e.g., 

In different cultures around the world, 
tome types of workers are always needed: 
L Workere to get food. ' 
2« Workers to make clothing* 
3. Workers to build homes. 



Social Studies: Answers questions based 
on material read. e.g.. We all know that 
different materials are uuid in construct- 
ing buildings. In what ways are these 
materials used? 
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CONTENT AREA AND TOPIC 



SKILLS AND KNOWLEDGE 
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Exposition 
PROCESSES 

Followinj; skills generally 
applicable to content areas, 
as well as to literature 
Comprehension 

Literal 



Factual materials give useful knowledge about the world. 



^^inferential^ 



Stated details in content area materials wmy be isolated 

as sub-headings in chapters or parts of chapters.**p 27 
The main idea of a selection My be directly stated in various 

locations within the selection. p.24 ^-"^us 
Clue words such as first , next , and last indicate 

sequential order of events. p. 28 
Clue words such as because and as signal cause and effect 

relations. p. 29 

Cause and effect relations not directly stated mv be deter- 
_^wined. p,31 



Critical 

Word Meaning 
Vocabulary Development 
Contextual Clues 

Structural Analysis 




/^^J-sntial order is a factor in predicting outcomes and draiT^ 
^ mg conclusions. p. 28 ) 

Experiential background and knowledge affect evaluation of 
written materials* p,34 



Vocabulary also expands through including content area words. 

Word meaning may be found from the context in a direct state- 
ment, through a synonym or through apposition. p. 22 

A prefix cor.es before a root word and changes the meaning of 
the word. p. 18 

A suffix cones at the end of a root word and changes the mean- 
ing of the word. p.l8 

superlatives change the meanings of adjec- 
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SKILL 



C Dctflmlnif^ Sequence 



LEVELB 



LEVEL C 



Deveiopt ihe undenttndinf that a 
aeries ofevents occuning within one 
episode may beordered sequentially. 



gMt^m^ Alts: Listens to a simple story 
and is guided to develop an understanding 
of sequence by answering such questions 
u: What happened first? What happened 
next? 

Social Studies: Places pictures in se* 
quential order to develop a simple time 
line. 

Science: Partic^ates in the writing of 
experience charts bated on firsthand ob* 
^rvations: How long is a day in fall, 
winter, and spring? 



Recognizes the relationship of se* 
quence to meaning. 



Language Aits: Reproduces story heard 
or read by drawing several consecutive 
pictures ofevents in sequential order. 

Science: Plans and lists (in proper 
quence) steps needed to .carry out a^ 
science experiment. 

Social Studies: Reads several sentences 
describing an event connected with the 
unit under study; arranges these in cor- 
.'^t order, e^g.. 

When people needed to travel, 
they used stagecoaches drawn by 
one or two horses. 

The first European settlers 
traveled from place to place on 
foot, on horseback, or in wagons 
drawn by horses. 

When the Indians lived in New 
York, they traveled on foot. 

Later, people traveled long dis- 
tances on trains urawn by steam 
engines instead of horses. 



LEVEL D 



Uses sequential order to make inte r- ^ 
pretations and prediction s; recog* 
nizes key vi^rds as clues to se- 
quential development 



) 



aiyiage Arts: Predicts outcome 



of stor y^^ 
[events; \ 



Stemming from loyal sequence of evenU 
picks out key words oc phrases which 
ht^i^t sequence, e;g., now, then, after 
a while, today, yesterday, tomorrow, in i 
[ie first place, in the sec ond place, etc y 

Science: Is able to renumber or rewrite 
in sequential order facts taken from text. 

Social Studies: Studies two pictures of 
related events in growth of towns, cities, 
or nations to determine time span and se- 
quence ofevents between them. 

Physical Activities: Gives facts sequen- 
tially when explaining how to play a 
game. 
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